System and method of categorizing communications

ABSTRACT

A system and method of categorizing communications involves evaluating one or more portions of a communication. Systems include a language module structured to analyze the one or more portions of the communication for at least one of a language characteristic or a non-word symbol within a text of the communication, an acoustic analysis module structured to analyze the one or more portions of the communication for an acoustic characteristic, and a determination module structured to determine at least one category label to associate with the one or more portions of the communication using at least one of the language characteristic or the non-word symbol, and the acoustic characteristic.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of the following U.S. patentapplication, which is incorporated by reference in its entirety: U.S.application Ser. No. 15/194,742, filed Jun. 28, 2016(CALL-0004-U01-001).

U.S. application Ser. No. 15/194,742 (CALL-0004-U01-001) is acontinuation of the following U.S. patent application, which isincorporated by reference in its entirety: U.S. application Ser. No.14/592,510, filed Jan. 8, 2015, now issued on Aug. 9, 2016 as U.S. Pat.No. 9,413,891 (CALL-0004-U01).

U.S. application Ser. No. 14/592,510 (CALL-0004-U01) claims the benefitof the following provisional applications, each of which is herebyincorporated by reference in its entirety: U.S. Application No.61/924,909, entitled “Real-Time Conversational Analytics Facility”,filed Jan. 8, 2014 (CALL-0003-P01) and U.S. Application No. 62/005,857,entitled “Real-Time Conversational Analytics Facility and Redaction”,filed May 30, 2014 (CALL-0004-P01).

BACKGROUND Field

This present disclosure relates to automated systems for monitoringcommunications, and more specifically to real-time automated monitoringsystems for monitoring and improving live communications, including byproviding feedback on communications performance.

Description of Related Art

Monitoring and improving the performance of a call center employee inhandling a call from a customer, typically a customer that has an issueor a problem, has always been a challenge. Supervisors may listen in ona call, and attempt to provide real-time feedback to the employee(hereafter referred to as a ‘call agent’, or simply ‘agent’), but giventhe number of concurrent calls coming into the call center it is notpractical to listen into but a very small number of calls. To handle thelarge number of concurrent calls, a post-call analytics system may beemployed that provides reports on the performance of the calls that havebeen recorded, but this does not help the agent during a particularcall, and thus only acts to provide overall “lessons learned” from anaggregate of calls and history thereof. Therefore there is a need forimproved methods and systems for monitoring and performance of livecalls, in real, or near-real time.

SUMMARY

In an aspect, a non-transitory computer readable medium with anexecutable program may provide a dynamic graphical representation of atleast one of a category, a score, a sentiment, and an alert. The programmay instruct a computer processor to perform at least the followingsteps: receiving a voice communication, analyzing the voicecommunication in at least near real-time relative to the receipt of thecommunication using a computer-based communications analytics facilityfor at least one of a language characteristic and an acousticcharacteristic, determining at least one of the category, the score, thesentiment, and the alert associated with at least one participant in thevoice communication using the at least one language and/or acousticcharacteristic, and providing a dynamic graphical representation of theat least one category, score, sentiment, or alert through a graphicaluser interface. The dynamic graphical representation may be provided toa user other than the participant in the voice communication as thereceived voice communication is on-going. The user may be anon-participant in the voice communication and the dynamic graphicalrepresentation is provided to assist the non-participant in at least oneof supervising and evaluating a participant in the voice communication.The dynamic graphical representation may be provided as feedback to aparticipant in the voice communication as the received voicecommunication is on-going. The acoustic characteristic may be at leastone of a stress of words, an aggregated stress of a plurality of words,an agitation, a tempo, a change in tempo, an amount of silence, asilence between words, a gain in volume or energy of the words, a tone,an overtalk, a time lag between words, a time dependency between keywords and phrases, an inter-word timing, an inflexion of words, and atemporal pattern. The language characteristic may be at least one of acategory, a sentiment, a regulation compliance, a score, a text, analternative text, a presence or absence of specific language, and aconfidence in word match. The steps may further include repeating thesteps for a plurality of in-progress voice communications, anddisplaying a dynamic graphical representation of each in-progress voicecommunication on the graphical user interface. The steps may furtherinclude assigning metadata representative of the voice communicationbased on the analysis, and displaying the metadata for the voicecommunication on the graphical user interface. Metadata representativeof the voice communication may include at least one of a speaker, anagent data, an agent grouping, a call handling location, a time and dateof call, a department, a skill or queue, a pertinent IVR path data, anda call length. The steps may further include before analyzing the voicecommunication for the at least one language characteristic, convertingthe voice communication to text using computer-based speech recognition.The analysis for the presence or absence of specific language mayinclude identifying whether a required statement has been spoken withina specific time period. The required statement may be a disclosure to aparticipant that satisfies a legal requirement, identifying the use ofprofanity, or identifying the absence of compliance scripts.

In an aspect, a product that provides a visual indication of a languageanalysis and an acoustic analysis of a communication may include areceiver for receiving a copy of the communication in near real-time orreal-time relative to the origination of the communication, acomputer-based communications analytics facility for evaluating thereceived communication in near real-time or real-time relative to thereceipt of the copy, including a language module for analyzing thecommunication for at least one of a language characteristic and anon-word symbol within the communication text that mimics a verbalelement, an acoustic analysis module for analyzing the communication foran acoustic characteristic, and a determination module for determiningat least one of a category, a score, a sentiment, and an alertassociated with the communication using the at least one languagecharacteristic, non-word symbol, and/or acoustic characteristic, and aninterface for displaying at least one visual indication representativeof the category, the score, the sentiment, or the alert. Thecommunication may be at least one of a voice communication, a voicemail, a chat, an email, a blog post, a survey, a social media post, anIVR session, and an SMS. The acoustic characteristic may be at least oneof a stress of words, an aggregated stress of a plurality of words, anagitation, a tempo, a change in tempo, an amount of silence, a silencebetween words, a gain in volume or energy of the words, a tone, anovertalk, a time lag between words, a time dependency between key wordsand phrases, an inter-word timing, an inflexion of words, and a temporalpattern. The language characteristic may be at least one of a category,a sentiment, a regulation compliance, a score, a text, an alternativetext, a presence or absence of specific language, and a confidence inword match. The visual indication may be at least one of a blinking, avibrating, a shaking, a pulsing, a movement, a color, a color change, acategory label, a score, and a sentiment. The analysis for the presenceor absence of specific language may include identifying whether arequired statement has been spoken within a specific time period,identifying the use of profanity, or identifying the absence ofcompliance scripts. The required statement may be constructed to satisfya legal requirement. The computer-based communications analyticsfacility may further include a metadata module for determining metadataof the communication. The metadata may be at least one of a speaker, anagent data, an agent grouping, a call handling location, a time and dateof call, a department, a skill or queue, a pertinent IVR path data, anda call length. The non-word symbol may be at least one of an emoticon, acapitalization, a punctuation, an ellipses, and a spacing. The at leastone category, score, sentiment, or alert may enable automaticallymonitoring at least one of a customer satisfaction, a compliance, anagent performance, and a performance indicator. The product may furtherinclude before analyzing the communication for the at least one languagecharacteristic, converting the communication to text usingcomputer-based speech recognition if it is a voice communication.Analyzing the text may include matching portions of converted text totext strings, wherein the text strings include key words and phrases.

In an aspect, a non-transitory computer readable medium with anexecutable program may provide a dynamic graphical representation of atleast one of a category, a score, a sentiment, and an alert. The programmay instruct a processor to perform at least the following steps:receiving a communication, analyzing the communication in at least nearreal-time relative to receipt of the communication using acomputer-based communications analytics facility for at least one of alanguage characteristic, a non-word symbol that mimics a verbal element,and an acoustic characteristic, wherein analyzing the languagecharacteristic of voice communications includes converting the voicecommunication to text using computer-based speech recognition,determining at least one of the category, the score, the sentiment, andthe alert associated with the communication using the at least onelanguage characteristic, non-word symbol, and/or acousticcharacteristic, and providing a dynamic graphical representation of theat least one category, score, sentiment, or alert through a graphicaluser interface.

In an aspect, a method may include receiving a telephonic voicecommunication, analyzing the voice communication in real time during thetelephonic voice communication using a computer-based communicationsanalytics facility for at least one of a language characteristic and anacoustic characteristic, determining at least one emotional stateindicator of a participant in the telephonic voice communication usingthe at least one characteristic, and providing a dynamic graphicalrepresentation of the at least one emotional state indicator through agraphical user interface. The dynamic graphical representation may beprovided to a user other than the first participant in the telephonicvoice communication as the received voice communication is on-going, theparticipant in the telephonic voice communication as the received voicecommunication is on-going, another participant in the telephonic voicecommunication as the received voice communication is on-going, or thelike. The user may be a second participant in the telephonic voicecommunication and the dynamic graphical representation is provided asfeedback to the second participant during the voice communication or anon-participant in the telephonic voice communication and the dynamicgraphical representation is provided to assist the non-participant in atleast one of supervising and evaluating a participant in the voicecommunication.

In an aspect, a method may include receiving a stream of acommunication, analyzing the communication in real time during thecommunication using a computer-based communications analytics facilityfor at least one of a language characteristic and an acousticcharacteristic, determining at least one of an emotional state indicatorof a first participant and a category of the communication using the atleast one characteristic, and providing a graphical user interface todisplay the at least one emotional state indicator or the category. Theat least one emotional state indicator or the category is provided to auser other than the first participant or the first participant in thecommunication as the communication is on-going. The user may be a secondparticipant in the communication and the emotional state indicator orthe category is provided as feedback to the second participant duringthe communication. The user may be a non-participant in thecommunication and the emotional state indicator or the category isprovided to assist the non-participant in at least one of supervisingand evaluating a participant in the communication.

In an aspect, a multi-platform customer input analysis system mayinclude a receiving module for receiving a plurality of heterogeneousinputs comprising at least one of a vocal communication and a textsource, a voice-to-text converter for converting vocal communicationinputs to text using computer-based speech recognition, an analysismodule for analyzing the text of each input source using acomputer-based communications analytics facility, a category facilityfor assigning a minimum of one category to each input based on the textanalysis, a scoring facility for assigning a minimum of one score toeach input based on the text analysis, and a storage facility forstoring at least one of the resulting scores, categories, input sourcetext, input vocal communications stream, text translation of the inputvocal stream, and input source type in a searchable database. The textsource may be one of chat, email, blog, survey, Facebook, Twitter,Google+, IVR, social media input, and SMS. The database may besearchable by a minimum of one of category, score, and input source. Thesystem may further include an acoustic analyzer for acousticallyanalyzing vocal communication input and assigning a minimum of onecategory to each vocal communication based on the acoustical analysis.The conversion of the vocal communications input to text may be doneonce and both the translation and the vocal communications input may bemade available for one or more of redaction, additional analysis,mining, and archiving in a searchable database.

In an aspect, a non-transitory computer readable medium may include anexecutable program for emotional analysis of text communications,wherein the program instructs a microprocessor to perform the followingsteps: receiving a text communication, analyzing the text communicationin real time for non-word symbols within the text that mimic verbalelements, and determining at least one emotional state indicator of thesender from the analysis. The steps may further include providing adynamic graphical representation of the at least one emotional stateindicator through a graphical user interface. Analysis of incoming textmay include analysis of non-word symbols such as emoticons,capitalization, punctuation, ellipses, and spacing. The steps mayfurther include storing the incoming text and the one or more emotionalstates in a searchable database.

In an aspect, a non-transitory computer readable medium may include anexecutable program for real time or near real time customer inputevaluation stored thereon, wherein the program instructs amicroprocessor to perform at least the following steps: receiving vocalcommunications from a plurality of heterogeneous sources, evaluatingsentiment of the vocal communications based on a plurality of acousticcharacteristics, assigning category metadata to the vocal communicationsbased on a plurality of acoustic characteristics, assigning scoringmetadata to the vocal communications based on a plurality of acousticcharacteristics, and storing at least one of the resulting scores,categories, input vocal communications stream, and input source type ina searchable database. The acoustic characteristics may include one ormore of silence between words, stress of words, aggregate stress of aplurality of words, tempo, changes in tempo, and gain of words. Theconversion of the vocal communications input to text may be done onceand both the translation and the vocal communications input may be madeavailable for one or more of redaction, additional analysis, mining, andarchiving in a searchable database. The medium may further includedisplaying visual indicators representative of one or more of thesentiment, category and score of the vocal communication.

In an aspect, a machine for automating support for a call center agentmay be clearly programmed to carry out at least the following steps innear real time: receiving a plurality of heterogeneous inputs comprisingat least one of vocal communications and text sources, analyzingacoustical characteristics of a vocal communication stream, assigning asentiment to a vocal communication stream based on the acousticalanalysis, assigning a score to a vocal communication stream based on theacoustical analysis, and storing a minimum of one of: resulting scores,sentiments, input vocal communication, and input source type in asearchable database. Analyzing acoustical characteristics of the vocalcommunication stream may include analysis of one or more of tempo,change in tempo, change in stress on words, overtalk, silence, and achange in volume.

In an aspect, a machine for automatically measuring agitation in nearreal time vocal communications may be clearly programmed to carry out atleast the following steps comprising: receiving a plurality ofheterogeneous inputs comprising vocal communications, evaluatingacoustic qualities comprising at least one of tempo, change in tempo,change in stress on words, overtalk, silence, and a change in volume,and assigning an agitation score to the call based on the analysis.

In an aspect, a machine for categorizing a plurality of heterogeneousinputs comprising at least one of vocal communications and textcommunications may be programmed to carry out steps comprising:providing an interface to facilitate a user in selecting categories tobe used for categorization of incoming communications, and providing aninterface to facilitate the user in specifying new categories to be usedfor categorization of incoming communications.

In an aspect, a machine for categorizing vocal communications may beprogrammed to receive a plurality of heterogeneous input vocalcommunications, analyze the acoustic characteristics of the vocalcommunication, convert the vocal communication to text usingcomputer-based speech recognition, analyze the text of the vocalcommunication using a computer-based communications analytics facility,store the text of the vocal communication in conjunction with theacoustic characteristics of the corresponding portions of the vocalcommunication in a searchable database; categorize the vocalcommunication according to the analysis, and search the database for atleast one of the acoustic characteristic and the text. The text analysismay include matching portions of converted text to text strings such askey words and phrases. Analyzing the acoustical characteristics of thevocal communication may include analysis of at least one of time lagbetween words, time dependencies between key words and phrases,inter-word timing, inflexion of words, gain in volume, and temporalpatterns. The analysis may be done in near real time or real time.

In an aspect, a machine for real-time quality monitoring of phone callsmay be clearly programmed to carry out steps: analyzing in-progresscalls for the presence or absence of specific language, analyzingin-progress calls for the presence or absence of specific acousticcharacteristics, further analyzing in-progress calls for the conjunctionof temporal and acoustic properties associated with the presence ofspecific language, providing alerts to at least one user based on theanalysis of the in-progress call, and enabling immediate, real-timereview of in-progress calls. The analysis for the presence or absence ofspecific language may include identifying whether a required statementhas been spoken within a specific time period, the use of profanity, orthe absence of compliance scripts. The required statement may be a “MiniMiranda.” The user may be at least one of a call-center agent and acall-center supervisor. The resulting analysis may be stored in asearchable database. The machine may further include calculatingcumulative scores based on analysis relative to target metrics ordisplaying cumulative scores and trends to individuals.

In an aspect, a product for real-time compliance monitoring of phonecalls may be clearly programmed to carry out steps including: analyzingin-progress calls for the presence or absence of specificcompliance-related language, analyzing in-progress calls for thepresence or absence of specific acoustic characteristics, analyzingin-progress calls for the conjunction of temporal and acousticproperties associated with the presence of specific language, providingan alert to at least one of a call-center agent and call-centersupervisor based on one or more of the presence or absence of specificcompliance related language in the in-progress call and the presence orabsence of specific acoustic characteristics in the in-progress call,and enabling immediate, real-time review of in-progress calls where oneor more of the presence or absence of specific compliance relatedlanguage and the presence or absence of specific acousticcharacteristics is detected.

In an aspect, a non-transitory computer readable medium with anexecutable program for facilitating the coaching of call-center agentsrelative to conformance with compliance regulations stored thereon,wherein the program instructs a microprocessor to perform at least thefollowing steps: analyzing in-progress calls for the presence or absenceof specific compliance related language, analyzing in-progress calls forthe presence or absence of specific acoustic characteristics, trackingthe results of the analysis for individual call-center agents, andproviding a tool for reviewing the performance of individual call-centeragents identifying opportunities for improved compliance withregulations.

In an aspect, a non-transitory computer readable medium may include anexecutable program for real-time quality monitoring of a plurality ofin-progress phone calls stored thereon, wherein the program instructs amicroprocessor to perform at least the following steps: analyzing aplurality of in-progress phone calls, assigning metadata representativeof the call based on the analysis, and displaying a visualrepresentation of each in-progress phone call on a screen where thevisual representation is representative of phone call metadata. Metadatarepresentative of the call may include at least one of category,sentiment, regulation compliance, tone, score, and agitation.

In an aspect, a machine for visually displaying relationships between aset of conversations may be clearly programmed to carry out stepsincluding providing identifiers for each of a set of conversations, andvisually displaying metadata common to two or more of the conversationswhere the visual display of the metadata provides information as to thefrequency of that metadata in the set of conversations being analyzed.The visual display may include sizing the metadata by frequency ofoccurrence in the set of conversations. The visual display may includerepresenting metadata in different colors according to their relativecorrelation.

In an aspect, a machine for automating support for a call center agentmay be clearly programmed to carry out the steps of: receiving a copy ofvocal communications from a plurality of heterogeneous sources in nearreal-time or real-time, analyzing received vocal communications in nearreal time for the presence or absence of specific language, the presenceor absence of specific acoustic characteristics, and the conjunction oftemporal and acoustic properties associated with the presence ofspecific language, and automatically providing at least one of data andscripts to an agent based on the near real time or real time analysis ofthe conversation. Analyzing received vocal communication in near realtime or real time may include one of: acoustically analyzing thereceived vocal communication including evaluating at least one of timelag between words, time dependencies between key words and phrases,inter-word timing, inflexion of words, gain in volume, and temporalpatterns, converting the vocal communication to text, and matchingportions of converted text to text strings such as key words andphrases.

In an aspect, a machine for automating a call center may be programmedto carry out the steps of: receiving a copy of vocal communications froma plurality of heterogeneous sources in near real-time or real time,analyzing a received vocal communication in near real time andautomatically providing a vocal response based on the real time or nearreal time analysis of the conversation. The step of analyzing a receivedvocal communication in near real time or real time may be executed on ananalysis module including: an acoustical analysis module for acousticalanalysis of vocal communication comprising evaluation of at least one oftime lag between words, time dependencies between key words and phrases,inter-word timing, inflexion of words, gain in volume, and temporalpatterns, a voice-to-text converter for converting the vocalcommunication stream to text, and an identification facility foridentifying the presence or absence of specific language to acousticcharacteristics.

In an aspect, a product that provides a visual indication of anacoustical analysis to a participant in the communication may include areceiver for receiving a copy of vocal communications from a pluralityof heterogeneous sources in real time or near real-time, an analyzer forevaluating received vocal communications in near real time including alanguage module for analyzing the presence or absence of specificlanguage, an acoustical analysis module for analyzing vocalcommunication comprising at least one of time lag between words, timedependencies between key words and phrases, overtalk, inter-word timing,inflexion of words, gain in volume, and temporal patterns, a sentimentassignment module for assigning a sentiment to the vocal communicationas a function of the acoustical analysis and the presence or absence ofspecific language, and a display for displaying at least one visualindicator representative of the real time or near real time evaluationof the vocal communication to one of the participants. The visualindicator may indicate the results of the real time or near real timeevaluation by one of blinking, vibrating, shaking, moving, changingcolor, and changing category label.

In an aspect, a machine for tracking all customer communication with anentity may be programmed to carry out the steps of: receiving aplurality of heterogeneous inputs comprising one of a vocalcommunication and a text source, assigning a customer identification toeach input, analyzing each input, assigning one of a category, score,sentiment, tone, compliance and agitation to each input based on theanalysis, storing the input, customer identification and one of score,category, sentiment, tone, compliance, and agitation in a searchabledatabase, searching the database by customer identification, anddisplaying all inputs associated with a customer identification. Theinput may be one of phone call, voice mail, VOIP, Chat, Email, SMS,Twitter, Facebook, Blogs and Surveys. Displaying all inputs may includedisplaying a chart of metadata over time.

In an aspect, a method of call classification capable of automated ABtesting utilizing historical data may include creating two or morecategories of calls, which differ in value for an identified piece ofmetadata, identifying similarities between calls within each group, andidentifying differences between groups.

In an aspect, a method of categorizing and storing customercommunications may include receiving a plurality of vocal communicationstreams from a plurality of heterogeneous sources, identifying patternswithin a vocal communication stream including: converting the vocalcommunication stream to text, matching portions of converted text totext strings such as key words and phrases, identifying recurring wordsand phrases not associated with existing key words and phrases,categorizing the vocal communication stream as a function of identifiedkey words and phrases, and creating new categories associated withrecurring words and phrases not associated with exiting key words andphrases.

In an aspect, automatically generating new categories for a customercommunications analysis system may include analyzing sets of archivedcall recordings, identifying a repeating word or phrase across aplurality of calls wherein the identified word or phrase is not apreviously known key word or phrase, generating a new category label tobe associated with the repeating word or phrase, and adding theidentified word or phrase and new category to means of categorization.

In an aspect, a non-transitory computer readable medium may include anexecutable program for a sortable database of acoustically analyzedcommunications stored thereon, wherein the program instructs amicroprocessor to perform the following steps including receivingincoming vocal communications, analyzing the acoustics of the incomingvocal communications, assigning one or more metadata values to theincoming vocal communications as a function of the acoustic analysis,converting incoming vocal communications to text, analyzing the texttranslation of the incoming vocal communications, assigning one or moremetadata values to the text translation, storing the metadata from theacoustic analysis and a minimum of one of incoming vocal communication,the text translation of the incoming vocal communication, customeridentification, agent, input source, time of day, location, and themetadata from the text analysis in a searchable database, sorting theentries in the searchable database based on the metadata from theacoustic analysis, and displaying a subset of the entries in thesearchable database as a function of the metadata from the acousticalanalysis. The entries may be sorted and displayed according to theconfidence level of the analysis.

In an aspect, a method for redacting numbers from a vocal stream mayinclude translating a vocal stream to text, analyzing the text toidentify the locations of numbers within the text, and redacting thenumbers from the vocal stream. Redacting may include one of removing,muting and bleeping. The vocal stream may be an incoming telephonicvoice communication or an archived voice recording. Translation of thevocal stream may be done using one of large vocabulary speechrecognition technology, phonetic engines, direct phrase recognition andcloud-based speech recognition.

In an aspect, a method for removing sensitive information from a vocalstream may include receiving a copy of the vocal stream, converting thevocal stream to text, analyzing the text and identifying sensitiveinformation in accordance with one or more rules, identifying thelocation of the sensitive information in the vocal stream, and redactingthe sensitive information from the vocal stream. The sensitiveinformation may be one of credit card number, security code, expirationdate, PIN number, date of birth, driver's license data, social securitynumber and financial account data.

In an aspect, a method of redacting incoming PCI sensitiveauthentication and cardholder data prior to recording may includecaching an incoming vocal stream in volatile memory such as RAM,converting the vocal stream to text, analyzing the text and identifyingsensitive information in accordance with one or more rules, identifyingthe location of the sensitive information in the vocal stream, redactingthe sensitive information from the vocal stream, and at least one ofstoring and transmitting the redacted vocal stream. In some cases, theremay be a redaction device attached to a recording system. The recordingsystem may store unredacted incoming vocal streams to volatile memory onthe redaction device. The unredacted vocal streams may then be redactedaccording to various methods described herein. In this way, theunredacted vocal stream need not be stored in non-volatile memory,facilitating compliance with various regulations on the handling of PCIdata.

In an aspect, a method of redacting incoming PCI sensitiveauthentication and cardholder data from an incoming vocal stream priorto receipt by an agent may include caching the incoming vocal stream,converting the vocal stream to text, analyzing the text and identifyingsensitive information in accordance with one or more rules, identifyingthe location of the sensitive information in the vocal stream, redactingthe sensitive information from the vocal stream, and transmitting theredacted vocal stream to an agent. The method may further includetransferring the unredacted incoming vocal stream to an alternate methodof receiving PCI sensitive authentication and cardholder data. 5-6seconds of the incoming vocal stream may be cached.

In an aspect, a machine for redacting PCI sensitive authentication andcardholder data prior to transmission may be clearly programmed to carryout the steps of the algorithmic method including identifying therecipient of an outgoing vocal stream, comparing the recipient to a setof approved recipients, for recipients not in the set of approvedrecipients, caching the outgoing vocal stream, converting the outgoingvocal stream to text using speech analytics technology, analyzing thetext and identifying sensitive information in accordance with one ormore rules, identifying the location of the sensitive information in thevocal stream, redacting the sensitive information from the vocal stream,and transmitting the redacted vocal stream to the unapproved recipient.

In an aspect, a machine for redacting PCI sensitive authentication andcardholder data prior to transmission may be clearly programmed to carryout the steps of the algorithmic method including identifying therecipient of an outgoing vocal stream, comparing the recipient to a setof approved recipients, transmitting the vocal stream to the approvedrecipients, for recipients not in the set of approved recipients,caching the outgoing vocal stream, converting the outgoing vocal streamto text using speech analytics technology, analyzing the text andidentifying sensitive information in accordance with one or more rules,identifying the location of the sensitive information in the vocalstream, redacting the sensitive information from the vocal stream, andtransmitting the redacted vocal stream to the unapproved recipient.

In an aspect, a system for redacting PCI sensitive authentication andcardholder data may include a data server deploying speech analyticstechnology, a hardware buffer for storing an incoming vocal stream, aspeech-to-text converter for converting the incoming vocal stream totext, an analysis module for analyzing the text and identifyingsensitive information in the vocal stream in accordance with one or morerules, a redaction module for redacting the sensitive information fromthe vocal stream, and a server for storing the redacted vocal stream.

In an aspect, a method for multi-level redaction of vocal streamsincluding caching an incoming vocal stream, converting the vocal streamto text, analyzing the text and identifying sensitive information inaccordance with one or more first rules, identifying the location of thesensitive information in the vocal stream, redacting the sensitiveinformation from the vocal stream, archiving the redacted vocal streamand the text, analyzing the text and identifying sensitive informationin accordance with one or more second rules, identifying the location ofthe sensitive information in the text, redacting sensitive informationfrom the vocal stream, and archiving the redacted vocal stream and text.

In an aspect, a method of multi-level redaction of vocal streams mayinclude caching an incoming vocal stream, converting the vocal stream totext, analyzing the text and identifying sensitive information inaccordance with one or more first rules, identifying the location of thesensitive information in the vocal stream, redacting the sensitiveinformation from the vocal stream, archiving the redacted vocal stream,analyzing the text and identifying sensitive information in accordancewith one or more second rules, identifying the location of the sensitiveinformation in the vocal stream, redacting sensitive information fromthe vocal stream, and archiving the redacted vocal stream and text.

In an aspect, a method of redacting PCI sensitive authentication andcardholder data from cloud based data may include converting thecloud-based vocal stream to text, analyzing the text and identifyingsensitive information in accordance with one or more rules, identifyingthe location of the sensitive information in the vocal stream, redactingthe sensitive information from the vocal stream, and storing theredacted vocal stream in a cloud-based storage system.

In an aspect, a non-transitory computer readable medium may include anexecutable program for redacting sensitive information from an incomingvocal stream prior to persisting incoming vocal stream to long-termmemory stored thereon, wherein the program instructs a microprocessor ofa recording device to perform at least the following steps: receive anincoming vocal stream, temporarily store the incoming vocal stream,convert the incoming vocal stream to text, analyze the text andidentifying sensitive information in accordance with one or more rules,identify the location of the sensitive information in the vocal stream,redact the sensitive information from the vocal stream, and persist theredacted vocal stream to long-term memory. The rules for identifyingsensitive information may include contextual analysis to identifyinformation protected by lawyer/client privilege or doctor/patientprivilege.

In an aspect, a non-transitory computer readable medium with anexecutable program for providing a dynamic graphical representation ofan emotional state indicator may instruct a microprocessor to perform atleast the following steps: receiving a voice communication, analyzingthe voice communication in real time using a computer-basedcommunications analytics facility for at least one of a languagecharacteristic and an acoustic characteristic, determining at least oneemotional state indicator of a participant in the voice communicationusing the at least one characteristic, and providing a dynamic graphicalrepresentation of the at least one emotional state indicator through agraphical user interface. The dynamic graphical representation may beprovided to a user other than the first participant in the voicecommunication as the received voice communication is on-going. Thedynamic graphical representation may be provided to the participant inthe voice communication as the received voice communication is on-going.The dynamic graphical representation may be provided to anotherparticipant in the voice communication as the received voicecommunication is on-going. The user may be a second participant in thevoice communication and the dynamic graphical representation is providedas feedback to the second participant during the voice communication.The user may be a non-participant in the voice communication and thedynamic graphical representation is provided to assist thenon-participant in at least one of supervising and evaluating aparticipant in the voice communication. The acoustic characteristic maybe at least one of a stress of words, an aggregated stress of aplurality of words, an agitation, a tempo, a change in tempo, an amountof silence, a silence between words, a gain in volume or energy of thewords, a tone, an overtalk, a time lag between words, a time dependencybetween key words and phrases, an inter-word timing, an inflexion ofwords, and a temporal pattern. The language characteristic may be atleast one of a category, a sentiment, a regulation compliance, a score,a text, an alternative text, a speaker, an agent data, an agentgrouping, a call handling location, a time and date of call, adepartment, a skill or queue, a pertinent IVR path data, a call length,and a confidence in word match.

In an aspect, a non-transitory computer readable medium with anexecutable program for real-time quality monitoring of a plurality ofin-progress phone calls stored thereon may instruct a microprocessor toperform at least the following steps: analyzing a plurality ofin-progress voice calls, assigning metadata representative of the voicecall based on the analysis, and displaying a visual representation ofeach in-progress voice call on a screen wherein the visualrepresentation is representative of phone call metadata. Metadatarepresentative of the voice call may include at least one of a category,a sentiment, a regulation compliance, a tone, a score, an agitation, atext, an alternative text, a speaker, an agent data, an agent grouping,a call handling location, a time and date of call, a department, a skillor queue, a pertinent IVR path data, a call length, and a confidence inword match.

In an aspect, a product that provides a visual indication of an acousticanalysis to a participant in the communication may include a receiverfor receiving a copy of vocal communications from a plurality ofheterogeneous sources in near real-time or real-time, and an analyzerfor evaluating received vocal communications in near real-timeincluding: a language module for analyzing the presence or absence ofspecific language, an acoustic analysis module for analyzing vocalcommunication comprising at least one of a stress of words, anaggregated stress of a plurality of words, an agitation, a tempo, achange in tempo, an amount of silence, a silence between words, a gainin volume or energy of the words, a tone, an overtalk, a time lagbetween words, a time dependency between key words and phrases, aninter-word timing, an inflexion of words, and a temporal pattern. Theproduct further includes a sentiment assignment module for assigning asentiment to the vocal communication as a function of the acousticanalysis and the presence or absence of specific language and a displayfor displaying at least one visual indicator representative of the nearreal-time or real-time evaluation of the vocal communication to one ofthe participants. The visual indicator may indicate the results of thenear real-time or real-time evaluation by one of blinking, vibrating,shaking, moving, changing color, and changing a category label for thevocal communication. The analysis for the presence or absence ofspecific language may include identifying whether a required statementhas been spoken within a specific time period. The required statementmay be a “Mini Miranda”. The analysis for the presence or absence ofspecific language may include identifying the use of profanity. Theanalysis for the presence or absence of specific language may includeidentifying the absence of compliance scripts.

In an aspect, a non-transitory computer readable medium with anexecutable program for emotional analysis of text communications mayinstruct a microprocessor to perform the following steps, including:receiving a text communication, analyzing the text communication in realtime for at least one of a language characteristic and a non-word symbolwithin the text that mimics a verbal element, and determining at leastone emotional state indicator of the sender from the analysis. The stepsmay further include providing a dynamic graphical representation of theat least one emotional state indicator through a graphical userinterface. The non-word symbol may be at least one of an emoticon, acapitalization, a punctuation, an ellipses, and a spacing. The steps mayfurther include storing the incoming text and the one or more emotionalstates in a searchable database. The language characteristic may be atleast one of a category, a sentiment, a regulation compliance, a score,a text, an alternative text, a speaker, an agent data, an agentgrouping, a call handling location, a time and date of call, adepartment, a skill or queue, a pertinent IVR path data, a call length,and a confidence in word match.

In an aspect, a non-transitory computer readable medium with anexecutable program for providing a dynamic graphical representation ofan emotional state indicator may instruct a microprocessor to perform atleast the following steps: receiving a voice communication, analyzingthe voice communication for at least one acoustic characteristic,converting the voice communication to text using computer-based speechrecognition, analyzing the text of the voice communication using acomputer-based communications analytics facility for at least onelanguage characteristic, determining at least one emotional stateindicator of a participant in the voice communication using the at leastone acoustic or language characteristic, and providing a dynamicgraphical representation of the at least one emotional state indicatorthrough a graphical user interface. The acoustic characteristic may beat least one of a stress of words, an aggregated stress of a pluralityof words, an agitation, a tempo, a change in tempo, an amount ofsilence, a silence between words, a gain in volume or energy of thewords, a tone, an overtalk, a time lag between words, a time dependencybetween key words and phrases, an inter-word timing, an inflexion ofwords, and a temporal pattern. The language characteristic may be atleast one of a category, a sentiment, a regulation compliance, a score,a text, an alternative text, a speaker, an agent data, an agentgrouping, a call handling location, a time and date of call, adepartment, a skill or queue, a pertinent IVR path data, a call length,and a confidence in word match. Analyzing the text may include matchingportions of converted text to text strings such as key words andphrases.

These and other systems, methods, objects, features, and advantages ofthe present invention will be apparent to those skilled in the art fromthe following detailed description of the preferred embodiment and thedrawings.

All documents mentioned herein are hereby incorporated in their entiretyby reference. References to items in the singular should be understoodto include items in the plural, and vice versa, unless explicitly statedotherwise or clear from the text. Grammatical conjunctions are intendedto express any and all disjunctive and conjunctive combinations ofconjoined clauses, sentences, words, and the like, unless otherwisestated or clear from the context.

BRIEF DESCRIPTION OF THE FIGURES

The invention and the following detailed description of certainembodiments thereof may be understood by reference to the followingfigures:

FIG. 1 shows a screen shot where it can be seen how the applicationassists in measuring and correcting hits, misses, false positives (FP),and the like.

FIG. 2A, FIG. 2B, FIG. 2C, and FIG. 2D depict embodiment dashboards.

FIG. 3 depicts an alternative embodiment dashboard.

FIG. 4A depicts an embodiment scorecard.

FIG. 4B depicts an embodiment of a personalized performance portal.

FIG. 5 depicts an embodiment high-level functional block diagram of theRT conversational analytics facility.

FIG. 6 depicts an embodiment command post with colored tilesrepresenting status of calls.

FIG. 7 depicts an embodiment detail for a call presented in the commandpost.

FIG. 8 depicts an embodiment call alert detail.

FIG. 9 depicts an embodiment user interface agent assistant.

FIG. 10 depicts a compliance user case summary.

FIG. 11 depicts a sales user case summary.

FIG. 12 depicts an escalations user case summary.

FIG. 13 depicts a customer retention user case summary.

FIG. 14 depicts an embodiment functional diagram for the intake portionof the RT conversational analytics facility.

FIG. 15 depicts a screenshot of a real time conversational analyticssummary page.

FIG. 16 depicts a screenshot of a score builder screenshot for theemotion score.

FIG. 17 depicts a screenshot of a score builder screenshot for thedissatisfaction score.

FIG. 18 depicts a screenshot of a score builder screenshot for the agentquality score.

FIG. 19 depicts a top-level process flow diagram for redaction.

FIG. 20 depicts a high level embodiment of rules of analysis.

FIG. 21 depicts a high level flow diagram of redaction.

FIG. 22A depicts a high level flow diagram of redaction.

FIG. 22B depicts a high level flow diagram of redaction.

FIG. 23 depicts a user interface for AB testing.

FIG. 24 depicts a user interface for a topicminer cloud.

FIG. 25 depicts a user interface for a topicminer cloud.

FIG. 26 depicts a user interface for searching communications.

FIG. 27 depicts a compliance summary screen.

FIG. 28 depicts a compliance summary screen.

FIG. 29 depicts a compliance configuration screen.

While the invention has been described in connection with certainpreferred embodiments, other embodiments would be understood by one ofordinary skill in the art and are encompassed herein.

DETAILED DESCRIPTION

The present disclosure describes a real-time (RT) conversationalanalytics facility that provides conversational analytics and real-time,or near real-time monitoring of communications from multiple channels,such as phone calls, chats, text messaging, blog posts, social mediaposts, surveys, IVR, e-mails, and the like, and may provide forfacilities that enable increased performance for individuals involved inlive conversational support functions (e.g. enterprise customer supportcall center employees). Real-time or near real-time may indicate thatthe system processes the conversations as they are received in order toprovide an output, such as immediate feedback, during the conversationas it is proceeding. Processing may be on the order of seconds,milliseconds, and the like. The real-time (RT) conversational analyticsfacility enables automatically evaluating in real-time or near real-timeevery communication (of heterogeneous types (e.g., voice, phone call[either to the call center or direct dialed to an agent], voicemail,chat, e-mail, blog, survey, Facebook, Twitter, Google+, other socialchannels, IVR, etc.) related to an activity, such as an activity of anenterprise, including in-progress calls, for sentiment/acoustics,categorization, and performance scoring, including the presence orabsence of specific language or acoustic characteristics, utilizingacoustic and conversational analysis to convert communications to a textformat for inclusion in a single database repository that includesfields for data relating to, at least, speech analysis/audio mining ofspoken communications. The communications may be routed to a centralizedhost or handled by distributed computing systems in various embodimentsdisclosed herein or via a cloud-based system. The raw, unstructured dataof recorded or real-time conversations is converted into consumable,structured data. Audio conversations are ingested by the system alongwith call metadata and speech-to-text transcription is performed togenerate a transcript that is analyzed using a set of linguistic andacoustic rules to look for certain key words, phrases, topics, andacoustic characteristics. Along with the call metadata, theseconversational and acoustic events allow the system to categorize andannotate the calls. In turn, the categories and events are incorporatedinto scores for each call, which can be used for automaticallymonitoring customer satisfaction, compliance, agent performance, and anynumber of customizable performance indicators.

With the basic conversational analytics system (text and speechanalytics), all contact points (recorded calls [that are speechrecognized], chats, SMS, social posts, blogs, surveys, etc.) areconverted into a consistent data structure/format so that they can allbe stored in a unified database. All data from multiple channels arecontained in the same database, and thus users can search, analyze, andreceive reports across these channels through the same interface. Forexample, if a user wanted to search for instances of escalation acrossmultiple channels, he or she simply needs to define the terms of thesearch through the main interface, and when the search is run, it willbring results from all forms of interactions. Similarly, users have theability to examine data from only one channel or a select number ofchannels. For example, FIG. 26 depicts search results for the keyword“baggage” and results from both web chat customer support and audiocalls come up in the search report. The conversational analytics systemallows for free form search to find and play back contacts containingcertain words and phrases or sharing other characteristics in a familiarweb search user experience. The conversational analytics system allowsfor evaluation and comparison of performance and key metrics using datavisualization. The conversational analytics system enables determiningroot cause through auto topic analysis and automatically identifyingoutliers. Output of text conversion/sentiment/acoustics analysis ofcustomer conversations and associated metadata from any sourcecommunication system (call recorders, chat systems, emails, social posts(e.g. twitter, Facebook), SMS, Blogs and Surveys, etc.) and acrossmultiple contact center sites and locations are stored in a database fora three step process that can be a cloud-based or an on premise callcenter monitoring solution that is scalable: (i) categorization; (ii)scoring; (iii) auto-analytics. A plurality of category labels areassigned for topics throughout the communication (e.g. Greeting,ID+verification, Behavior/emotions, Procedure discussed, Competitor,Churn, Likelihood for upsell/cross-sell, Agent performance, Customersatisfaction, Credit card (mention CCV code), custom categories, customsets of categories, search-based categories). The system's sophisticatedcategorization capabilities are significantly more powerful than simpleword and phrase spotting, with the ability to identify more complexlanguage patterns. The ability to weight various discrete searches andset category thresholds allows both false positives and false negativesto be minimized. Category labels enables automatic topic identificationfor any freeform defined set of calls. Terms can be retrieved in amatter of seconds from literally millions of hours of an indexeddatabase. The weighted sum of categories may be used to determine ascoring. A weighted aggregation of some categories may be performed.Scores may be tied to different goals, such as customer satisfaction,agent performance, etc. These and other systems and methods will befurther described herein.

Conversational analytics may provide facilities that enable performanceevaluation, determination of call contact influence factors, how toimprove ‘first-call’ resolution, trend tracking and analysis, ad hocsearching, discovery, root cause analysis, handling efficiency, businessintelligence, competitive intelligence, and the like. Real-timemonitoring may provide facilities that enable monitoring, alerting,decision support, next-best-action guidance, script adherence, offerassistance, supervisor shadowing, monitoring of escalation, callintervention, course correction, immediate review of occurrences ofevents and infractions such as escalation attempts, the use ofprofanity, the absence of compliance scripts (e.g. has a mini-Mirandawarning been delivered by a particular time or point in the call?), andthe like.

The RT conversational analytics facility may provide for the capture ofmetadata by a recording system, including third-party systems. Inaddition, the RT conversational analytics facility may generate acousticmetadata, such as including stress of words, aggregated stress of aplurality of words, agitation, tempo, change in tempo, amount ofsilence, silence between words (implied distance between communicationand response), gain or energy of the words, overtalk, and the like, ofrecognized speech or non-audio on each communication channel. Forinstance, the RT conversational analytics facility may include metadatato be captured such as text, alternative text (as determined by variousprocesses, such as the one described in U.S. Pat. No. 7,346,509, whichis incorporated by reference herein in its entirety), speakers, agentdata (e.g., a unique identifier or agent name), agent grouping (e.g., away to group agents for review purposes), location (e.g., where a callwas handled), time and date on call capture (e.g., exactly when the callwas captured by the recording system), department, skill or queue,pertinent IVR path data, call length, silence time or percent (e.g.,largest block of silence in seconds), silence between words (e.g.implied distance between communication and a response), agitationaggregate, tempo, sentiment, category, score, confidence (confidence ofthe likelihood of acoustic match and the weight of the model based onprevious word/context), aggregated confidence, gain or energy of thewords, and the like. In non-audio communication, the analysis ofstress/agitation may include analysis of emoticons, capitalization,exclamation points, punctuation and other non-word symbols that mimicverbal elements.

In embodiments, acoustically analyzed communications may be included ina sortable database. The database may be sorted to display the entrieswith the highest confidences for improved readability, instead of arandom display.

Metadata in the system may be provided, such as providing tags or names,such as naming employees, supervisors, groups of users, and the like.For example, the system might refer to agents as ‘Team Members’. Withthis, a name or handle may be substituted for the underlying datarepresentation, where these names persist throughout the user interface.In addition to adding discrete and continuous data during each contact,topics, categories, scores, and the like, may be ‘tagged’ to a call. Thesystem may provide for a default set as well as providing customization.

The RT conversational analytics facility may utilize large vocabularycontinuous speech recognition (LVCSR), non-dictionary dependent system,or other speech recognition engines, such as those detailed for speechanalytics (http://en.wikipedia.org/wiki/Speech analytics). Although LVCSR is used herein as an illustrative speech recognition engine, oneskilled in the art would recognize that other speech recognition enginesmay also be applied to the embodiments described herein, such asphonetic engines, direct phrase recognition, cloud-based speechrecognition (such as SpeechAnywhere from Nuance), and the like. Due tothe large variations in conversation topics and in speaker voices, thespeech recognition engine should preferably obtain the optimal accuracyfor word and topic identification. Language modeling may be used toinform recognition. The RT conversational analytics facility may ingestvoice through a single pass upon receipt, where upon various data isgathered. These may include all the words that are spoken in the call,as well as acoustic information such as the level of stress in thevoices, the amount of silence in the conversation, the rate of speech,overtalk, and the like.

Continuing to utilize a LVCSR speech recognition engine as anillustrative example, the LVCSR may perform not only a phonetic indexingof the data, but also may apply a statistically weighted language modelto the sequences of phonemes to derive the most likely combination ofphonemes into words (e.g., the sounds “Z” and “K” are rarely heard inthat order) and sentences of what was actually being said (e.g., tosolve for acoustic confusion, such as for example “recognize speech”versus “wreck a nice beach”). This is similar to how the human brainapplies context to reason out of what is being said in a noisyenvironment. For example, if you are in a loud doctor's office, thenurse most likely said “Wait Here”, not “Weight Hear”. The RTconversational analytics facility may use a large collection of data,such as thousands of hours of conversational corpus to derive thesemodels, such as utilizing the computational power of many servers totake the place of painstakingly slow manual proofing that is required byseparate linguists using a phonetics-based solution. In the case ofusing a many-server solution, the investment in hardware to perform thisautomation may be initially slightly more than a phonetics solution, butthis automation may lead to a lower overall total cost (TOC) (e.g.,trading expensive phonetic/linguistic analysts for off-the-shelfhardware) and a quick payback on business intelligence. The increasedaccuracy allows the RT conversational analytics facility to derivevaluable business intelligence from the audio in a matter of minutes orless, rather than the days it takes a human analyst to review the mostlikely phonetic candidates and conduct linear searches on the phoneticstream for matches, thus improving the overall speed and efficiency ofthe computer for analyzing conversations.

The comparison of ‘phonetics’ and ‘LVCSR’ is not merely comparing onetechnique that uses phonetics and one that does not. All LVCSR acousticmodels are phonetically based. In both approaches, each phoneme ismodeled in its phonetic context (e.g. the “t” between “s” and “r” in“street” is different acoustically than the “t” between the start of aword and “ah”, as in “top”). While phonetic approaches do take intoaccount what the possible sequences of sounds are and their frequency(for example, groups of consonants, such as “stldr” never occur inEnglish), they do not typically take into account any higher-levelknowledge of the language. In contrast, in an LVCSR approach, the largercontext that the sounds occur in is taken into account as well, so inbuilding a lattice of alternatives, only sequences that occur in realwords in the language are included, and the likelihood of one word tofollow another is taken into account in how likely each alternative is.This compensates for the fact that the sounds are very ambiguous andtend to merge with neighboring sounds (e.g. “dish soap”) and the samesequence of sounds can be different word sequences: “let us pray” vs.“lettuce spray”. The LVCSR approach algorithmically determines whichalternatives are likely in the language, letting the computermathematically sift through the results quickly and efficiently, ratherthan requiring a significant manual effort. The knowledge of what wordsexist, and are likely in the target language, is a major factor inhigher accuracy.

An LVCSR-based RT conversational analytics facility may use alexicon/dictionary with a large vocabulary, such as 50,000 words. Theintroduction of new words is fairly rare with a vocabulary of this size.The system may on occasion update the dictionary if a new word isrepeated consistently across the tens of thousands of hours that thesystem mines daily. For those words not in the language model, arecognizer may choose a word that is phonetically similar. For example,in audio that has been processed containing mentions of a specificstring such as the Verizon product term ‘FIOS’, the recognizer mayconsistently choose the word ‘files’. Since the RT conversationalanalytics facility allows users to see the resulting recognitionresults, this may be easily found in the data. An alias rule may be setup so that in category definitions created to track mentions of the‘Verizon FIOS’ product term, the underlying search could utilize thephrase “Verizon files” as a proxy for the term “Verizon FIOS.”

In embodiments, recognition may occur as streamed recognition, withrecognition of a word at a time versus snippets at a time until comingupon a trigger/viable context time. Contextual look-ahead may improvelatency. For example, with respect to the word bankruptcy, the streamingrecognizer hears “ . . . ruptcy” after “Bank . . . ”. Contextually, thephrase “I'd like to declare . . . ” may be a signal. So if the streamedrecognizer recognizes “declare . . . ” followed by “Bank . . . ”, thenthe recognizer may contextually look ahead and determine that it wouldsee “Bankruptcy”, thereby reducing recognition latency.

Confidence of acoustic match may be calculated and updated throughoutthe communication. In embodiments, the RT conversational analyticsfacility may calculate a confidence level in the translation of the callwhere the confidence level may represent an aggregation of theconfidence levels for individual words within the call. Confidencelevels for individual words may be based on the likelihood of anacoustic match and the weight of a context model based on neighboringwords/context for individual words. The confidence level for individualwords may be updated throughout the call before being finalized at theend of the call.

The RT conversational analytics facility provides advantages forconversation analytics, including allowing the process to be handled bycomputers rather than relying solely on human resources (e.g., somesolutions only organize and structure data for easier human review and‘analytics’, where the RT conversational analytics facility is capableof creating full transcription, categories, and quantification of allcontact data and alerts without human intervention), providing anarchitecture that enables fast response times (e.g., because the RTconversational analytics facility may recognize and retain all the wordsin the conversation, the response time for users working with theapplication is fast, where other systems retain only phonemes ratherthan words, such that converting phonemes into words typically causessystem performance and analyst productivity to suffer), true discovery(e.g., allowing for both automated and human discovery of trends orissues that users would otherwise not know to look for, such as ‘havingall of the words’ in the data allowing for frequency and clusteringanalysis that is not feasible in other solutions), and the like.

With respect to data sources, the conversational analytics system maywork with many data sources, regardless of the specific call recordingsystem or combination of call recording systems, or sources of textcontacts (e.g., voice, text, email, SMS, chat, blog, social networkposts, and the like). The open adapters of the RT conversationalanalytics facility may integrate with multiple third party systems,within a single installation, and the like, to extract the contacts andassociated descriptive data demographics (metadata) for processing. Ifan existing recording system produces speaker separated audio (stereo),the RT conversational analytics facility may identify the individuals inthe conversation in all interfaces, and allow them to be analyzed andsearched separately. The RT conversational analytics facility mayinclude a cloud analytics and interceptor recording option for voice,where the fidelity of the audio surpasses any proprietary on-premisesrecorder option, resulting in unprecedented word transcription accuracy.The RT conversational analytics facility interceptor may record TDM andVoIP traffic in a speaker separated manner, such as an exact digitalcopy of the original conversation with no loss in quality. TheInterceptor may be configured to record the IVR portion of the call andtreat it as a separate segment. In an embodiment, recording can continuewhile one side is on mute or on hold.

In embodiments, a customer “call” center system may comprise a pluralityof channels for receiving heterogeneous customer input to an enterprise,such as: phone calls to a call center; phone calls to an agent; voicemail; on-line chat; e-mail; blogs; surveys; Facebook; Twitter; Google+;other social channels; IVR; and the like. Customer input received as avocal stream may be translated to text using a variety of speechanalysis techniques such as phonetics, LVSCR, and the like. Thetranslated text may be analyzed and the text categorized and scored. Thecustomer input received as text may be analyzed and the text categorizedand scored. A single searchable database may be used to store all of thereceived customer input such as: vocal stream as received; texttranslation of the received vocal stream; text as received;categorization; scoring; system metadata such as source of customerinput, customer identification, agent ID, and the like; and the like.The database may be searchable by category, score, input source, and thelike. In embodiments, database content may be displayed in an orderedmanner where the order may be determined based on the value of ametadata category such as: alphabetically by category; ranked high tolow by score. For customer input initially received as an input vocalstream the metadata used for sorting may relate to the translation ofthe customer input from vocal stream to text such as: confidence levelin the translation; scores analysis related to auditory processing; andthe like.

In embodiments, the recognition and analytics need be only done once tobe useful for real-time analysis & monitoring, redaction and/or mining.In embodiments, storing the translation of the input vocal stream mayeliminate the need to translate the vocal stream in the future foradditional analysis. In embodiments, a multi-platform speech recognitionsystem may facilitate real time vocal analysis and translation of vocalstream from a plurality of sources to text. The redaction of sensitiveinformation, categorization of incoming vocal stream, scoring ofincoming vocal stream, and data mining may be done in conjunction withthe translation of the vocal stream to text or at a later date. Storingthe text translation of the vocal stream may eliminate the need totranslate the vocal stream again in the future should additionalanalysis be of interest.

With respect to accuracy, simple transcription accuracy may be avariable based on a number of influencing factors such as quality of therecording, mono versus stereo recording mode, voice transmission type(i.e. SIP or TDM), headset quality, accents, and the like. Accuracylevels in optimum conditions may be upwards of 95%, where the typical“transcript only” accuracy is typically more in the 55-60% range. Sincethe transcript is only the first step in the process, the key tosubstantially higher levels of interaction interpretation accuracy iscentered on a next step in the analytics process, namely categorization.Categorization is a method of tagging a conversation that containscertain topics or characteristics, and the like. The RT conversationalanalytics facility's categorization capabilities are significantly morepowerful than simple word and phrase spotting, with the ability toidentify more complex language patterns. The ability to weight variousdiscrete searches and set category thresholds may allow both falsepositives and false negatives to be minimized. Accurate categorizationis required for accurate analytics. Use of the RT conversationalanalytics facility provides a tool that is more accurate than countingsearch hits by using categories defined by multiple weighted criteriaand advanced language logic that maximizes hits while reducing falsepositives. This provides accuracy in categorization achieved despite thereality of imperfections in speech recognition. The system's automatedcategorization is more accurate and consistent compared to manualapproaches, which translates into, for example, a 50% word accuracyachieving a greater than 80% categorical accuracy. The experience (e.g.,such as through the generation of a large corpus) and learned modelingallows users to benefit from libraries of existing vertically targetedcategories (e.g., 300 categories, 5400 patterns).

Categorization is a component of voice of customer conversationalanalytics (speech analytics, text analytics). Categorization is theautomatic tagging of contacts that contain certain language patterns,keywords, phrases, or other characteristics. Any given customer contactis likely to be tagged as belonging to several categories. Categoriesenable a user to find, count, and trend contacts that contain thesecharacteristics. In an embodiment, a common taxonomy of category groupsmay be used when tagging calls and text communications within contactcenters across different industries. One category may be behaviors, forexample, how agents or customers are behaving. For example, arecustomers expressing dissatisfaction, and is there an empatheticresponse to that dissatisfaction. Various language patterns, keywords,phrases, or other characteristics associated with the overall feel of‘dissatisfaction’ may be included in a list for the category. When thelisted item appears in the communication, the ‘dissatisfaction’ tag maybe applied.

One category may be reasons, such as the reason for the contact, invoice often referred to as the call driver. For example, a customer maycall their bank for a balance inquiry, and as a follow up the agent mayconduct a transfer—each of these would be a reason for the call. Onecategory may be procedures, such as whether or not agents areappropriately complying with procedures. This category may be commonlyused in collections to ensure agents are saying things they should sayand not saying things they shouldn't, according to FDCPA (or FSA)regulations. One category may be outcomes, such as measuring theresponse to specific actions. For example, how a customer responded toan upsell offer. One category may be products, such as whether or notcertain products are mentioned. One category may be competitors, such aswhether or not certain competitors are mentioned. Other categories mayinclude dissatisfaction, empathy, repeat contact language, transferlanguage, politeness, and the like.

Category groups and definitions can be configured, created, andorganized to suit a user's needs. The same methodology and often thesame language patterns can be used across communication channels forconsistent means of analysis and lower overhead in category definition.Categorization is optimized for big data, using simple, consistentalgorithms regardless of the communication channel, while stillsupporting sophisticated language patterns. The language patterning iscapable of measuring language in specific locations of conversation,order of occurrence of language, positive or negative rules, standardBoolean logic, and the like. In embodiments, categories may becustomizable so that users can edit what they want to be as the languagepatterns, keywords, phrases, or other characteristics that should giverise to a particular label.

In embodiments, analysis may include acoustical elements such as timelag, time dependencies, inter-word timing, inflexion of words, gain involume of words or series of words, and the like to provide additionalcontext to the identification and categorization of specific words,phrases, and Boolean combinations thereof. In an illustrative andnon-limiting example, a lag or change in emphasis between words maychange the meaning of the phrase, indicating sarcasm, irony, and thelike. In one example, the two phrases: “you know dude” and “you know . .. Dude?” may be different. The first example may be rhetorical and thesecond a request for confirmation that the other party is present. Inembodiments, the acoustical analysis may provide information onsentiment, agitation, and the like.

In embodiments, new categories may be automatically created over time.Differences and similarities between sets of calls, where callers arediscussing a topic in one set but not the other, such as differences inphrases, words, acoustic characteristics and the like may be analyzed tolook for indicators that a new category may be appropriate. A machinefor evaluating archived call recordings, where the machine is clearlyprogrammed to carry out the steps of the algorithmic method, may be usedto analyze archived call recordings. The analysis may compriseidentifying repeating similarities in words, phrases, or acousticcharacteristics that are not already associated with a category. Theidentification of a new pattern of repeated similarities may indicate anopportunity for a new category. In embodiments, the new category may bederived from a repeating word or phrase, acoustic characteristic, or thelike. In an illustrative and non-limiting example, an airline callcenter may have existing categories such as “delayed flight”, “cancelledflight”, and the like. However, a volcanic eruption, such as that inIceland in 2010, may result in the sudden appearance of words such as“volcano”, “ash cloud” and the like, appearing in calls categorized as“delayed flight” and “cancelled flight.” A new category may be addedcalled “volcano.” This new category may facilitate distinguishing callsassociated with the volcanic eruption from routine calls categorized as“delayed flight” or “cancelled flight.”

In embodiments, there may an ability to mine archived calls to identifythe root topics for a given set of conversations. A user may be able todefine a set of conversations based on one or more of a plurality ofcharacteristics such as category, score, sentiment, agent, location,event attributes, call center, time of day, and the like. An analysis ofall the words contained in the user-defined set of conversations may beperformed and commonalities between the conversations identified. Thismay facilitate discovering additional common topics or event attributesassociated with these conversations without any need for pre-definitionof topics. Referring to FIG. 24, an illustrative and non-limitingexample of a screenshot 2400 is shown comprising a user input interface2402 and an output display of common topics 2404. The user inputinterface 2402 facilitates the identification of a set of conversationsbased on certain common characteristics. The resulting output display ofcommon topics 2404 may provide insight into topics or characteristicsthat are more or less common among the selected set of conversations.The degree of commonality may be expressed as an ordered list, by fontsize of characteristic, as shown in FIG. 24, color, and the like. Thismay facilitate insight into commonalities in a set of conversations. Inembodiments, changes in trends across all measures, categories andattributes may be identified. Outliers within a conversation set may behighlighted in comparison to a chosen baseline. This may facilitate userinsight regarding where to pay attention and what is significant. Theoutput display may be shown either as a tree view or as a tag cloud.Both provide a quick visual understanding of what is happening in thecommunications with no pre-conceived notion (or scripted query) of whatto look for. The topic cloud can be alphabetically ordered, sized byfrequency, colored by correlative relationship to each other, and thelike. TopicMiner is a clustering engine that takes all the wordscontained in a set of conversations defined by the user, and presentsvarious topics that drive these conversations. In essence, the systemautomatically discovers the nature of these conversations for the userwithout any need for pre-definition of topics. Referring to FIG. 24,users can even target results by speaker, location, and various eventattributes. This allows users to get to the facts and contacts thatreveal the issues within a conversation set rather than having to siftthrough each and every contact. In this example, one call is clicked onto ‘Get Topics’ and then a topic cloud is formed.

Referring now to FIG. 25, a search for “harass*” revealed many topicsassociated with communications that included the keyword “harass*”. Oncea topic cloud is formed, one can click a word from the cloud and thecall list is filtered to obtain just the calls to which the topicbelongs. In this example, the category “Wrong number” is being clickedto reveal those communications that include the phrase “harass*” and thecategory “Wrong Number”. The topic cloud may also be modified to onlyinclude topics from the subset of calls. Thus, root causes, or insightsyou didn't know to look for are revealed through TopicMiner™.

The RT conversational analytics facility may provide fullconfigurability of categories to the users in a friendly intuitivesearch interface. Each category search predicate may be weighted andrefined with others so that ‘context’ can be achieved (e.g., a 10 digitnumber near the phrase “area code” is most likely a telephone number).False positives generated by events such as IVR and hold messages fullof product mentions or customer service tactics (e.g., “please wait”)are removed by either excluding the specific offending phrase from theaggregated list of results or by requiring additional evidence that thisis indeed the topic being searched for (i.e. other topic mentions inclose proximity, specific location in a call such as in the first 30seconds, or speech recognition confidence thresholds). In addition, theRT conversational analytics facility may contain a set of pre-configuredbehavior categories that are common across all contact centers. Thesebehavior categories may be tuned by identifying categories that areunder or over-hitting on a specific phrase or topic, such as because ofa matching script, IVR message, and the like, and narrowing searchcriteria to a more likely set of topic mentions. Finally, the hugestatistical models of the LVCSR engine may ensure that false positivesgenerated by the recognition process itself are minimized, especially incomparison to a phonetics engine. As new categories are created, or aswords are added to the vocabulary, the data can simply be re-processed.Referring to FIG. 26, a search of communications for the term “baggage”reveals results from multiple channels of communications. The resultscan be sorted along multiple dimensions, such as confidence, score,duration, date, and the like. A filters tab may reveal scores percategory for the communication(s) in the search results. Results may bedrilled down by clicking on the particular category to review only thosecommunications that have the keyword and the category from the filter.In an embodiment, outliers within each conversation set may behighlighted in comparison to a chosen baseline.

In an example, a customer service quality package may provide a standardset of ‘instant insight’ categories and scores to help measure theperformance of agents and an associated contact center. The RTconversational analytics facility is relevant to a broad range ofindustries and includes a plurality of metrics, categories, scores, andthe like. For instance, categories may include agent ownership, churnlanguage, compliments, dissatisfaction, empathy, escalation, holdlanguage, payment language, politeness, repeat contact, sales language,transfer language, understandability issues, and the like. Scores mayinclude agent quality, customer satisfaction, emotion, and the like.Other metrics may include contact duration, percent silence, longestsilence, tempo, word count, acoustic agitation, and the like.

In an example, a sales effectiveness package may provide a standard setof instant insight categories and scores to help measure the performanceand effectiveness of sales personnel (e.g., agents in a contact center).For instance, categories may include sales ownership, assumptive close,upsell/cross sell, touting benefits, objection handling, do not call,dissatisfaction, payment language, politeness, empathy, hold language,transfer language, understandability issues, and the like. Scores mayinclude sales effectiveness, customer satisfaction, emotion, and thelike. Other metrics may include contact duration, percent silence,longest silence, tempo, word count, acoustic agitation, and the like.

In an example, a collections compliance risk mitigation package mayprovide a standard set of Instant Insight categories and scores to helpmeasure and monitor compliance, such as the collections effectiveness ofagents. Based on specific compliance rules, the RT conversationalanalytics facility automatically monitors in-progress calls, utilizingthis package, for the presence or absence of specific compliance-relatedlanguage. With the broadening focus of the Consumer Financial ProtectionBureau (CFPB), and other federal, state and local officials on thecollections industry, being able to effectively monitor complianceacross all contacts is a requirement. This solution package may berelevant to first and third party collections agencies, financialservices and other or organizations with collectors in their contactcenter. For instance, this package may include categories such as rightparty contact mini-Miranda, Fair Debt Collection Practices Act (FDCPA)violations, legal action mentions, dissatisfaction, risk & abusivelanguage, wrong number, request for payment, promise to pay, refusal topay, collector ownership, politeness, empathy, hold language, transferlanguage, understandability issues, and the like. Scores may includecompliance risk, collector effectiveness, hot calls, and the like. Othermetrics may include contact duration, percent silence, longest silence,tempo, word count, acoustic agitation, and the like. Thus, based onspecific compliance rules, in-progress calls may be automaticallymonitored for the presence or absence of specific compliance-relatedlanguage to provide alerts and enable immediate review of occurrences ofevents.

In embodiments, a search QA may be provided that is a specific tool thatadvanced analysts can use to build and test categories. Referring toFIG. 1, a screen shot is presented where it can be seen how theapplication assists in measuring and correcting hits, misses, falsepositives (FP), and the like.

With the ability to analyze and examine each contact, the RTconversational analytics facility may allow for automated scoring ofcontacts/communications, such as every call center contact. Scores maybe created using a mixture of categories, acoustic measures, filters,attributes, and the like. This mix of tools allows analysts to tailorscores to fit the specific needs of an organization. These scores may beprovided prominently to a user through the user interface. Scores forcalls and non-audio contacts may be continually updated throughout theinteraction. Once individual contacts are scored, they can then beaggregated and combined across different timeframes to identifyperformance trends. Scoring allows users to aggregate any number ofmetrics including category hits or weights, acoustic measures, andmeasures ingested as metadata, into a performance index that can beassessed against every contact. This data is then sent to the user UIpage to provide direct and continuous feedback. When creating a score,Acceptable Ranges may be set in order to identify which score valuescorrespond to good, average, and poor performance. These ranges may beused to provide feedback on performance across various timeframes.

Scores may be the most meaningful and useful information for effectiveconsumption by business users. They are the basis for tracking trends,measuring improvements in performance, comparing peers, and the like.Scores may enable the organization to implement key performanceindicators (KPIs) relating to customer conversations.

In embodiments, a score builder application may allow a user to edit orbuild scores in an intuitive and easy-to-use interface with a widevariety of configuration options. Scores may be assessed against anycontact, or targeted to specific contacts defined through filtering(e.g., using metadata, attributes, acoustics, or categories). Forexample, a score may be assessed against contacts handled by certainagent groups, of a certain length, that contain escalation language, andthe like. Configuration of weights for each component of a score mayprovide the control to fine-tune a score and to match existing manualperformance measures, quality programs, and the like.

In embodiments, automated call center performance management withscoring may be provided. Scores help a user track performance trendsover time and compare relative performance across different groups ofcontacts (e.g., such as agent groups or teams). A user may have theability to define thresholds or acceptable and unacceptable ranges forscores, with the ability to label and color-code each range. Labelingand coloring target ranges may simplify the analysis of scores in heatmaps, scorecards, or performance portals. Sometimes a grade, descriptivelabel, or color is more telling than the underlying number value. If thecall recording platform has ability to do speaker separation, such aswhich dictates a “stereo recording” mode, then the system may be able todetermine emotional scoring differences between the caller and theagent.

Scoring enables the aggregation of any number of metrics includingcategory hits or weights, acoustic measures, and measures ingested asmetadata, into a performance index that can be assessed against everycontact. This unique ability to score every contact is the heart of theautomated quality monitoring and performance management solution, butcan be used to measure any key aspect of business performance such asagent quality, sales effectiveness, contact efficiency, compliance risk,customer satisfaction, network reliability (in communications), and thelike. A Score Builder application and method enables editing or buildingscores in an intuitive and easy-to-use interface with a wide variety ofconfiguration options. Scores can be assessed against any contact, ortargeted to specific contacts defined through filtering (using metadata,attributes, acoustics, or categories). For example, a score can beassessed against contacts handled by certain agent groups, of a certainlength, and that contain escalation language. Configuration of weightsfor each component of a score provides control to fine tune scores andto match existing manual performance measures or quality programs.Scores enable tracking performance trends over time and comparingrelative performance across different groups of contacts (such as agentgroups or teams). Thresholds may be defined or acceptable andunacceptable ranges for scores may be defined, with the ability to labeland color code each range. Labeling and coloring target rangessimplifies the analysis of scores in heat maps, scorecards, or varioususer interfaces of the system. Action may be taken automatically basedon the score, such as with a coaching tool. For example, for an agentscoring low in customer satisfaction, politeness training may bedelivered automatically. Effectiveness of the training may be evaluatedby re-scoring subsequent to the training.

Referring now to FIG. 15, a summary page of scores generated by the RTconversational analytics facility for a single call is shown. In thisview, measures, such as percentage silence, date/time of the call, theduration, and an indication of agitation are shown. Various indexscores, such as emotion score, agent quality score, agentquality—customer service, customer satisfaction, and compliance risk areshown. Various other index scores, individual behavior scores or othermeasures may be shown in this view. In this example, it can be seen thateach index score comprises one or more individual scores that contributeto it. In this example, the emotion score is an aggregate of thedissatisfaction score, the escalation score, and the agitation score. Inthis example, the Agent Quality—Customer service score is an aggregateof the appropriate opening score, politeness score, agent ownershipscore, percentile silence score, proper hold procedure score, andempathy score. In this example, the agent quality score is an aggregateof the politeness score, complements score, percentage silence score,empathy score, and agent ownership score. In this example, thecompliance risk score is an aggregate of the right party contact riskindicator, mini-Miranda risk indicator, and FDCPA risk indicator.

Referring now to FIG. 16, a view of the score builder application isshown. The score builder displays settings for which individual scoresare aggregated to generate an index score. In this example, the emotionscore is shown and the individual scores, or indicators, that areaggregated to generate the emotion score are agitation score,dissatisfaction score, escalation score, and negative emoticon.Indicators may be added or subtracted by the user. For each indicatorscore, a point range is given within which the individual score will beused in the calculation of the index score. In this example, the pointrange is 0 to 100 for each of the individual scores, however, it shouldbe understood that any point range may be used in the calculation of theindex score. For example, the user can modify the settings so that onlypoints scores in a narrow range, say 60 to 100, are used in thecalculation of the index score, and if the points fall outside of therange, the score is ignored for the purposes of calculating the indexscore. The point value may indicate how many times a particular phraseor a particular word was uttered during the conversation. For eachindividual score, there may be particular phrases or words that arecounted. The phrases and words used for each individual score may becustomizable by the user. In other embodiments, the score may incrementon the basis of volume, tone, silence, occurrence of a specific event,and the like. FIG. 17 shows a listing of words and phrases that arecategory hits for the dissatisfaction score. For each time the word orphrase indicated in the ‘Name’ column is uttered, a count is incrementedfor the dissatisfaction score. Additionally, a weight may be applied tothe counts for the uttered words and phrases when generating the overalldissatisfaction score. In this case, the weight is 1 for each of thewords and phrases, however, it is understood that any weight can beapplied to the words or phrases and the weight may be user customizable.For example, if the phrases ‘angry about’ and ‘Board of Directors’ wereuttered during the conversation, the dissatisfaction score would be 2.In the example of FIG. 16, the dissatisfaction score is 0 indicatingthat none of the words or phrases listed in the behavior category ofdissatisfaction was uttered. In some embodiments, the individual scoreis not calculated based on words or phrases uttered, but on the lack ofwords or phrases uttered. For example, a percentage silence score may becalculated based on the actual percentage of time that silence wasdetected during the conversation. As with the other scores that arebased on the accumulation of detected words or phrases, the percentagesilence score may also be weighted. Further, the weight may be usercustomizable.

Continuing with the example of FIG. 16, a weighting may also be used inthe calculation of the index score. Once the individual indicator scoresare calculated, a weight may be applied in the aggregation of theindividual indicator scores to generate the index score. In thisexample, the dissatisfaction, escalation, and negative emoticon scoresare more heavily weighted than the measure of agitation (which itself isa measure derived from the change in tempo, change in stress, andvolume) in the calculation of the emotion score. If the settingsdepicted in FIG. 16 aside from the negative emoticon score are used inthe calculation of the emotion score shown in FIG. 15, thedissatisfaction score would be weighted by 2/5, the escalation scorewould be weighted by 2/5, and the measure of agitation would be weightedby 1/5. Thus, the dissatisfaction and the escalation scores are 0, andthe measure of agitation of 24.09 is divided by 5 to achieve the finalemotion score, which is 4.8, rounded up to 5. For each index score, theaggregation method may be sum or average. There may be settings relatedto the index score. For example, particular ranges of scores may beassociated with a description. In the example of FIG. 16, a scorethrough 79 indicates the suspected emotion, a score from 79 to 80indicates normal emotion, and any score above 80 indicates calm. Theuser may define any number of ranges. Each score range may also beassociated with a particular color when displayed to a user. At leastone score range made be identified as the target range outside of whichan alert may possibly be generated. The user may indicate if the emotionscore should be displayed as a numerical score, as the text description,as a color, some combination of any of the above, or the like.

Referring now to FIG. 18, another example of the score builder for theagent quality score is shown. In this example, the indicators arepercentage silence score, politeness score, empathy score, agentownership score, and complements score. When aggregated to generate anagent quality score, in this example, the percentage silence score isweighted by 3/14, the politeness score is weighted by 3/14, the empathyscore is weighted by 2/14, the agent ownership score is weighted by5/14, and the complement score is weighted by 1/14. Further, in thisexample, there are 3 defined ranges of agent quality score that areassociated with a description. In this example, scores below 35 indicateneeds coaching, scores between 35 and 75 indicate meets targets, andscores above 75 indicate outstanding. Each score range may also beassociated with a particular color when displayed to a user, at leastone score range made be identified as the target range, and the user mayindicate if the score should be displayed as a numerical score, as thetext description, as a color, some combination of any of the above, orthe like.

The RT conversational analytics facility may examine the emotionalcontent of a conversation from a variety of factors. For auditory data,the RT conversational analytics facility may measure emotion based on acombination of acoustic characteristics of each speaker's voice as wellas the context of the conversation. These acoustic characteristics mayinclude the amount of physical stress in the voice, the changes in thestress, as well as the rate of speech. All three of these components maythen be combined based on a weighted average into a measure calledagitation. Agitation by itself may be purely a measure of the volatilityof a speaker's voice throughout a conversation. This volatility could bethe result of an extremely negative event or an extremely positiveevent. The system may then take that measurement and combine it with thecontext of the conversation, which is revealed through the speechrecognition engine. A final emotion score may be assigned to each callbased on the aggregate of the agitation measure as well as the contextof the call. A low emotion score may be associated with negative emotionand a high emotion score may be associated with positive emotion. Fortext data, a similar process may be used to generate an emotion scorefor each side of a conversation, but since text data does not containacoustic characteristics, the system may measure other forms ofemotional expressions, such as emoticons, capitalization, spacing,punctuation such as ellipsis, exclamation marks, and the like.

Each conversation, whether verbal or non-verbal, may be associated withan emotion score, and the score may be available for any type ofanalysis offered by the user interface. These may include trend analysisof emotion scores over time, comparison of emotion scores across variousaggregations based on metadata, as well as inclusion of the emotionscore in the construction of other measures.

Referring to FIGS. 2A, 2B, 2C, and 2D, example dashboards are presentedthat focus on the emotion score measure, including a trend graph (FIG.2A), comparisons of the score across various aggregations/measures (FIG.2B), Percent Analysis (FIG. 2C), a listing of contacts (FIG. 2D), andeven the ability to pull up the best scoring (or worst scoring) callsdirectly on the front page of the application. The dashboard overviewmay provide an at-a-glance high-level review of the metrics and trendscompared across data segmentation, such as type of contact, contactdriver, agent group, agent, and the like. Each user, based on theirinterests and objectives, may dynamically configure the overview.Parameters of the charts can be changed on the fly, allowing the quickvisualization of different information and comparison of data across themultiple charts. For example, for a trend analysis graph of compliancerisk, if the chart showed a particularly high risk on a particular day,the data point for that day can be clicked on to drill down and obtaindetails of the communications that gave rise to the high compliancerisk. An interface, such as the interface shown in FIG. 26, may revealthe search parameter (e.g. the selected time frame), may showtranscripts of communications from that time frame, data on the scoresper category for communications or communication sets, and the like.

Referring to FIG. 3, another style of dashboard is presented to examinescores, the explorer. Explorer provides a ‘heat-map’ style summary ofany score and its contributing indicators or score components. Scoresmay be grouped dynamically across any sub-categories. From the Explorer,users may be able to increase detail to include all underlying detailand effectively determine the root cause and record corrective orcoaching actions.

Referring to FIG. 4A, with the score card, multiple scores and theircontributing indicators may be viewed within a single page view that isdesigned for distribution to a person, group, department manager, andthe like. A comparison for performance across scores and indicators maybe drawn against a user-selected entity, such as another agent, group,or otherwise defined set of contacts. Performance feedback is deliveredas continuous, plain language alerts and notifications, personalized foreach user (individuals or team leaders) so that agents or supervisorscan take direct action from feed items. Action may be takenautomatically such as by delivering politeness training automatically toan agent in need, grading the agent's effectiveness, and then re-scoringthe agent after training.

FIG. 4B displays another example of a personalized performance portalthat provides contact center managers, supervisors, and agents directaccess to comprehensible and continuous feedback, driving sustainableperformance improvement. From this view, any individual agent can beselected so that all of the data pertains to that agent only. Agents orsupervisors may have access to this view, so agents may be encouraged tocompete beyond just hitting the minimum target since access to rank datafor each metric and key performance driver motivates a competitiveculture for high performance.

The Metric Tiles 402 are easy to read performance indicators thatdisplay current performance & trend information, and compare scores fromthe timeframes selected on the page. The score shown in the metric tileis generated using available contacts from the current timeframe foreither internal data and/or third party data. If applicable, the metrictile also indicates whether a metric has improved or declined since thecomparison timeframe when compared to the current timeframe. To viewmore details and filter the page by a metric, the corresponding metrictile may be selected. Metric tiles may aggregate automated scores,manual scores, & other key performance metrics from disparate systems.Metric tiles may enable quick interpretation of performance through boxand value colors with configurable ranges. Users may customize whichmetrics to display for each team/group. Metrics may be displayed asnumbers, labels, or ratings (80, A+, Exceeds Expectations, ««««, etc.).Customers can customize “weighting” for each metric to their specificpriorities and goals. Selecting which metrics to display is simple, suchas by drag and dropping a metric from the Available Metrics column tothe Selected Metrics column. Only metrics that contain data for the lastmonth will be displayed in the Available Metrics column. Forconvenience, the Available Metrics column is sorted alphabetically. Oncein the Selected Metrics column, a user may indicate whether they wouldlike to display the metric numerically or as a descriptive label. Theorder of the metrics in the Selected Metrics column dictates the ordermetrics are displayed on the user interface. The Selected Metricscolumns may be re-ordered by dragging scores up or down. To remove ametric from the Selected Metrics column, the corresponding ‘X’ may beselected; the metric should reappear in the Available Metrics column. Tosave the settings, the ‘Save’ button at the bottom of the page may beselected.

Metric Details 404 are visible when the page is filtered by a metric.Metric details Displays the metric score for the current timeframe alongwith an improving/weakening trend indicator. Also included in the MetricDetails box is a sparkline graph. The sparkline operates independent ofthe page's timeframes, and is instead generated using data that isavailable for a time period. The sparkline allows a user to monitor themost recent performance for the selected metric, and can be added to aWatch List by clicking on the Watch link to its right.

The Bar Chart 408 is visible when the page is filtered by a metric. Thebar chart is a visual representation of score data for the selectedtimeframes. The first bar represents the entity's current score value,while the next two bars provide comparisons against the entity's peers.The vertical line indicators on each bar denote the entity's previousscores according to the comparison timeframe. The bar chart's backgroundcolors correspond to the scoring ranges setup for that metric.

Scorecard 410 provides a complete breakdown of the selected score. Eachscore component is displayed along with its corresponding sub-score. Inaddition to providing an inside view for the selected metric,comparisons against past performance and that of peers is provided. Forease of use, information is presented in both text form and graphically.

Rank List 412 is visible when the page is filtered by a metric. The ranklist displays the immediate child entities of the currently selectedentity. Child entities are ranked according to score. The default viewsorts entities on the list by score in ascending order. Child entitiescan be added to a Watch List directly from the Rank List by clicking onthe supplied Watch links.

News Feed 414 displays several different types of items, from ‘Did YouKnow’ alerts, to action items from a supervisor. The news feed generatesnews items using the latest available data, so users can react andadjust performance as necessary.

Watch List 418 provides a snapshot of recent performance. Watch list maynot be affected by the page timeframes, but rather may always displaydata for a predetermined time period. By monitoring the sparklines inthe watch list, trends in recent performance may be identified and, ifnecessary, adjusted accordingly. Wherever there is a Watch link, it canbe clicked to add the metric to the watch list. Individuals can keep aneye on key areas at risk or needing improvement through a personalizedlist.

Underlying contact information may be accessed readily from any of thesedashboards. Views may be published or subscribed to for recurringdistribution to an email account and to a selected schedule. Inaddition, a full set of predefined standard management reports may bedeveloped through reporting services.

The RT conversational analytics facility may measure the amount ofsilence in a call at ingestion time. The process may involve looking forperiods of silence, such as longer than 1.3 seconds long, and each ofthese segments may be captured individually. This granularity may allowthe system to capture silence information in high detail. Users may beable to see the overall amount of silence within a call, the placesduring which silence segments occurred, the average length of thosesegments, the number of those segments, and the like. At a more advancedlevel, users may perform analysis on content before and right aftersilence segments to determine the reasons for silence. Silence measuresmay be present and available for use by analysts at differentinteraction levels. Silence may be used to define searches, incorporatedinto scores, used as aggregation for comparison of groups, used increation of conversation sets for ad-hoc analysis, and the like. Themeasurement of silence may lead to determination of issues that can betackled immediately upon the implementation of the system.

The RT conversational analytics facility may offer a comprehensive setof compliance features. The RT conversational analytics facility mayhave standard solutions that monitor every call, such as for MiniMiranda language, Right Party Contact language, FDCPA violations,abusive language from either party, and other risky language. Every callmay be scored to identify relative risk level associated with an aspectof compliance based on content of the conversation. This may allow fornarrowing corrective action efforts and investigation where they areneeded most. Call analysis may occur in near real-time and tagging orindexing of violations within contacts may allow for immediatenavigation to the occurrence of the violation. The RT conversationalanalytics facility may enable the ability to tag calls that are in orout of compliance, quickly conduct ad hoc searches for language,navigate directly to potential compliance breaches, produce complianceaudit scorecards, pinpoint high risk teams and agents, and the like. Theresult may be lower cost and effort in monitoring, faster responsetimes, and ultimately reduction and/or elimination of fines or lawsuitsfor non-compliance with CFPB regulations and the FDCPA.

For instance, working in the debt collection and financial spaces thesystem may be able to fine-tune a baseline content to give clients asignificant edge from day one. This content may include compliancerelated to the Mini Miranda, Right Party Contact, FDCPA, risksassociated with legal action mentions, and the like. This content may beapplied both from a post-call perspective to highlight non-compliantconversations as well as from a real-time perspective, to ensure thatcalls are proceeding in a compliant fashion. Supervisors may be alertedto situations where compliance was not met, and review and remedy orintervene in those calls. In addition, from an analytical perspective,the RT conversational analytics facility may help a client understandthe compliance levels of various agents and agent teams and pin-pointpotentially problematic areas. Referring to FIG. 27, two illustrativeand non-limiting examples of compliance summary screens 2702A 2702B areshown together with an example of a detailed drill down 2704 showing thespecifics of the conversations aggregated in the compliance summaryscreens 2702A 2704. Referring to FIG. 28, another illustrative andnon-limiting example is shown of a compliance summary screen 2702Cdisplaying a list of call agents ordered by compliance risk. Althoughretrospective views are shown in FIG. 27-FIG. 28 it should be understoodthat this analysis may be done in near real-time or real-time onin-progress calls. In addition to the default content, the system mayalso provide users with the ability to define additional content relatedto more specific disclosures or other compliance items. These pieces maybe easily configured via a standard user interface, and provide theability to track both the presence as well as absence of specificcompliance items. Referring to FIG. 29, an illustrative and non-limitingexample of a compliance configuration screen 2902 is shown which mayfacilitate the ability to configure the compliance monitoring facilitythrough such features as: adding additional content to be monitored forcompliance; selecting which content is monitored; identifying action tobe taken on identification of content associated with compliance; andthe like.

With RT conversational analytics facility, a user may be able toautomatically analyze a contact to protect privacy and monitor privacycompliance policies, review for explicit compliance language, such asFDCPA/CFPB and CCA requirements for credit discussions, monitor 100% ofprotected class discussions, forensically discover fraud and abuse riskand research events, automatically redact transcripts and mined audiofor private information, such as social security numbers, credit cardnumbers, and the like, to ensure PCI compliance, and the like.

In embodiments, for integration, the RT conversational analyticsfacility may need access to at least two items: the calls/recordings andthe metadata. Depending on the recording system, these can be retrievedin multiple ways. For recordings, they can come from a file share, froma web service, or sometimes from an API call. Metadata usually comesfrom the recordings system's database, but can also be ingested as XMLfiles or CSV files. Some systems may involve database integration forthe metadata. Recordings may be pulled from the file store of therecording system while ongoing calls may be analyzed in the cloud or asthey are being recorded.

The RT conversational analytics facility may comprise severalcomponents. For example, on the backend, a site manager may beresponsible for interfacing with various audio and text sources toprepare content for mining. One audio format may be PCM WAV (8 khz, 16kbps) but other supported audio formats include those playable inWindows Media Player (AIF, AIFC, AIFF, ASF, AU, AVI, MOV, MP2, MP3, MP4,MPEG, MPG, MPG4, QT, WMA, WMV), WAV (linear PCM, 8 KHz, ulaw, A-law,GSM, MP3, G.726, G.729 encoding), VOX (ADPCM encoding), NICE.nmf., andthe like.

A group of pipeline servers may then handle the processing. The sitemanager may control the assignment and assembly of information to andfrom these pipelines. For data storage, the system may utilize SQLServer and has multiple primary databases: The operational database maycontain all mining results; the logging database may keep track ofactivity; the pipeline database may be used by a site manager to trackmining operations, the reports database may contain the interface forthird party systems to extract data, and the like. For the front end,the system web and search API may host the applications, a mongo serverto host the database, and SQL reporting services to support reportcapabilities.

In embodiments, once a call/email/chat is available for the RTconversational analytics facility to ingest, the processing time may bedependent on the current contact volumes and the amount of hardwaredeployed. In some configurations, data may be available within seconds,while in other configurations with less processing capability, data maybe available within, for instance, 5-15 minutes and 30-60 minutes duringpeak times. This may all be dependent on a queuing mechanism that is,for example, first in first out (FIFO). The system may support otherqueuing mechanisms that are designed to avoid FIFO queue starvation forhigh volume peaks. Pre-emptive FIFO may allow a portion of newest callsto the front of the line to ensure that backlog calls and new calls arebeing processed together. Audio may be cached and may include, forinstance, 90 days of audio and word retention, with 13 months ofbusiness data retention (e.g., scores, category hits). In apremise-based environment, database storage may be a matter of storageallocation, such as configured at approximately 1 gb for each2,000-2,500 hours of voice recordings. All original audio may be deletedon ingestion into the application. Only redacted audio may be stored forplayback. Foreign languages may remain transcribed in their originallanguage and not converted to English, however the categorization andscorecards may be converted to English.

With the RT conversational analytics facility, a user may gain theability to quickly distribute this data to individuals at severaldifferent levels of the organization, leveraging the power ofvoice-of-the-customer analytics. By dispensing this data throughout theorganization, the user may be enabled with information that allows themto improve performance, efficiency, decision-making, and the like.Typical distribution may be by email list. A report may be delivered onetime or on a recurring subscription basis allowing a user to get regularupdates for tracking and monitoring a specific metric or problem. A usermay use a send report feature to automatically e-mail a report toindividual users or groups. In addition, instead of emailing reports toa large distribution group, a user may be able to save the reports in ashared folder on the network. As long as the person viewing the reporthas access, the reports will open. This is an efficient way of managinga large volume of reports that are intended to be shared by a largegroup.

The RT conversational analytics facility offers real-time speechanalytics on top of the foundational analytics structures describedherein, which may be applied, for example, as a call center monitoringsystem, and may include a command user interface for an agentsupervisor, administrator, agent monitor, and the like. Although a callcenter monitoring system is used herein, one skilled in the art willappreciate that the system may be applied to other systems where livemonitoring of a caller or participant in a conversation is to bemonitored to improve the ability for a receiver of the call to providefeedback in handling of the call, and additionally, for monitoringsupervisors to more fully and actively assist the receiver, as well asmonitor overall performance (e.g. for a plurality of call channels). Inaddition, and continuing with the call center example, an agentassistant for the individual agents may be provided, such as forflexibility and customization of scripts, campaigns, timing, acousticaltuning, weighting and fly-outs for script prompting and adherence, andthe like.

The RT conversational analytics facility may provide features that allowboth supervisors and agents to monitor live calls. On the supervisorside, a supervisor user interface for all ongoing agent calls may bedisplayed to a supervisor, and he or she can then easily track thegeneral sentiment of each call as well as be alerted to adverse events(such as noncompliance or escalation). On the agent side, the system mayprovide an application that sits on the agent's screen, and providesfeedback as the call is ongoing, which may be based on the categoriesand scoring described herein (e.g., an agent assistant or ‘mood ring’that graphically displays attributes for the caller, such as in colorcode, text, blinking, vibrating, shaking, category label, and the like),such as described in FIG. 9, FIG. 10, and FIG. 11. The application maybe set to provide reminders for agents to give mandatory disclosures,inform them if such actions have taken or not taken place, and the like.The system may also provide for advice to be displayed, such as in agraphical “fly-out” of the application, as certain language is detectedon the call. For example, if the customer asks to “speak to asupervisor”, the application can pop out a small reminder to the agenton the proper escalation procedures. There is no limit to the amount ofdifferent fly-outs that the system can configure into the application,and setting them is as easy as identifying the language that wouldtrigger each fly-out, and the content of these fly outs. Since thecontent is driven from a central server onto the agent desktopapplications, a user may only need to perform these content updates atone location. The agent assistant may be configured to show fullycustomizable guidance and alerts based on a variety of businessscenarios. Agents may be provided next-best-action guidance in the formof links to relevant information or advice when certain items ofinterest occur on the call, for example: a technical issue with aproduct or a specific objection to an agent offer.

In embodiments, one or more screen shots of an agent's screen may becaptured at different points during a call and stored in a searchabledatabase together with a plurality of information about the call such asa recording of the call, categories assigned to the call, scoresassigned to the call and the like. In embodiments, it may be possible tolink a screen shot to a location in the call recording or texttranslation at which the agent would have been viewing the saved screenshot. In embodiments, the agent may select to initiate a screen shot. Inembodiments, a screen shot may be initiated based on one or more of aplurality of metadata related to the call such as categorization,scoring, key words, sentiments, and the like.

The RT conversational analytics facility may benefit applications,including increased sales, improved legal compliance, increased customersatisfaction, and the like, thus maximizing performance, such as inindustry contact centers. For instance, the RT conversational analyticsfacility may enable increased contact center agent performance inreal-time while calls are still ongoing, such as to ensure positiveoutcomes. The RT conversational analytics facility may optimizereal-time agent performance with state-of-the-art speech technology,monitoring ongoing calls in the call center, sending critical real-timealerts to supervisors, providing timely next best action guidance toagents, and the like. From the supervisor user interface command post,supervisors may be provided a view of key events on calls in progress,such as including customer sentiment, escalation attempts, complianceviolations, sales opportunities, and the like, where the command postmay automatically prioritize high-value call events, giving supervisorsthe opportunity to adjust performance in real-time (e.g., by making acrucial adjustment to the progress of the call). Based on conversationalflow between agents and customers, the call agent user interfaceassistant may produce timely, relevant information and pivotal guidanceto agents. For example, constant feedback of customer agitation may keepagents mindful of talk-down opportunities. Procedural scripts,qualifying upsell offers, or context driven guidance, may beautomatically offered at critical moments, ensuring agents followcorrect procedures and stay compliant with complex regulations.

FIG. 5 provides a high-level functional block diagram of the RTconversational analytics facility, where the system takes the rawunstructured data from conversations, and turns it into consumablestructured data. Real-time speech analytics analyzes live audio forongoing conversation, and produces alerts and guidance (e.g., withinseconds) of key events occurring on the call.

The process starts by ingesting audio conversations along with a richset of call metadata, and performing full speech recognitiontranscription, such as based on large vocabulary speech recognitiontechnology or other speech recognition technology. Those transcripts arethen analyzed by a sophisticated set of linguistic and acoustic rules,looking for key events and topics of business value. Along with the callmetadata, these conversational and acoustic events enable the system toautomatically categorize and annotate each call. In turn, thosecategories and events may be incorporated into scores for each callcontact, which may be used for automatically monitoring customersatisfaction, compliance, agent performance, and the like. The endresult is a wealth of structured data ready for analysis, automatedquality, performance management scorecards, real-time supervisor andagent alerts, and the like.

In embodiments, this wealth of structured data may be used to facilitateAB testing. In embodiments, the system may facilitate the identificationof two sets of data (A and B) having clear differences in one or moremetadata values of interest such as agent, overall call performance,successful upselling, differences in script, differences in shift andthe like. The rich metadata associated with the calls of each set maythen be mined for similarities between calls within a set. Theseidentified similarities within a set may then be compared withidentified similarities in the other set and differences identified.Referring to FIG. 23, an illustrative and non-limiting example of ascreen shot 2300 is shown comprising a display of the characteristicscommon 2302 within each of the two different sets of conversations. Inan illustrative and non-limiting example, one set of agents (A) may begiven one script, Script 1, from which to work while a second set ofagents (B) may be given a different script, Script 2, from which towork. The metadata associated with the calls completed by agents A and Bmay be analyzed and similarities and differences between the two groupsidentified. If all the calls completed by agents in group A are rankedhigher that those completed by agents in group B where the consistentdifference is the script used, this information may be used tofacilitate the selection of a script for future calls. In anotherillustrative and non-limiting example, calls may be divided into twogroups based on successful upsell. The metadata associated with thecalls may be analyzed and similarities and differences between the callsidentified to see if there are certain scripts, agents, acousticcharacteristics, and the like, which appear to correlate to successfulupselling. This information may then be used to facilitate agenttraining for successful upselling. In some embodiments, it may bepossible to complete the analysis on in progress calls as well as thosein a recording archive. For example, communication scoring can becompared between agents who might have done actions/said somethingdifferently from their agents. Based on the testing, the system maysuggest target phrases for future communications. In FIG. 23, the userhas defined two conversation sets A and B, where set A represents callsduring which successful sales are made, and set B represents those withunsuccessful sales. The discover analytics automatically processes thedata from each set and compares them against a baseline to highlightwhat topics and characteristics drive A and what drives B. Users canthen easily spot the key trends that drive a successful sale versus anunsuccessful one. This methodology can be applied to any segregation ofdata, and allow users to obtain valuable insights in near real-time.

The system may provide for a supervisor's user interface, also referredto herein as the ‘command post’. The supervisor's user interface mayprovide for headline status and table view of a plurality of callagents. Calls may be sorted by overall status by default, such as withidle agents at the bottom, or other sorting criteria. There may be aninterrupt column that allows the supervisor to flag calls (e.g., withcolor-coded priority) that have been interrupted or escalated forinterruption. There may be a status column, such as for the status ofcurrent calls, calls that have been interrupted, and the like. There maybe a ‘talk-idle time’ indicator that tracks duration of the call orbetween calls. There may be an agitation indicator showing the currentlevel of agitation on each call, the highest level of agitation for eachcall, and the like. Violations may be selectable to access snippets,events, and the like.

Referring to FIG. 6, the command post may show colored, or otherwisedifferentiable, tiles, each representing a live ongoing call from one ofthe agents in an agent group that are based on auto-analytics. In thisrepresentation, the green tiles represent calls that are going well, andthe yellow tiles show calls that may be of some interest. The red tileshave been determined by the system to be calls that need immediateattention. The tiles may be organized alphabetically, by priority, andthe like. When a live call goes through the system it appears on theinterface, and dependent upon what language is used and the acousticcharacterization of the voices in the conversation, the tile is, in thisinstance, color coded and correspondingly prioritized. Language used onthe call may include key words and phrases that cause call priority toescalate in real-time from green, to yellow, to red, such as thesequence: “Hello, I have an issue I need to resolve” (tile starts outgreen), “Please let me speak to a supervisor” (the tile transitions toyellow, in response to the language and/or acoustic characteristics ofthe caller's voice), and finally “This customer service is ridiculous,and I'd like a full refund of my money” (red, in response to a stillmore agitated language and/or acoustic character of the caller's voice).All of the agent tiles may be sorted by severity, size, and/or color toindicate the status so that a user using the command post can quicklysee which agents should be monitored. In this dashboard view of FIG. 6,the tiles are adapted for easy visualization, such as to be big,rectilinear, color block, white text, including relevant data with size,color and placement to provide a visualized score or work flowprioritization. Supervisors can keep an eye on key areas at risk orneeding improvement through a personalized list of agents. A socialconstruct (including a social feed) may enable commenting on theinsights from auto-analytics and highlighting changes in the analyticflow. The comments may be promoted/sorted based on an “interestingnessalgorithm” which may account for seniority of commenter, frequency ofcomments, extent of agitation, and the like. Any user of the system maybe able to comment on any of the automated feeds anywhere in the system.The comments themselves may be analyzed for the same acoustic valuesused to score communications. For example, a bad comment about an agentmight rise to the call center manager.

Referring to FIGS. 7 and 8, a detail of the call is shown, such asprovided in response to selecting the tile of interest. From thisdetail, the supervisor may be able to view each of the events on thecall, the time they occurred, the text of what was said at the time, andthe like. Other actions the supervisor may be able to perform from thisinterface is to play back each of the call snippets, play back the fullcall anywhere from the beginning to the current time in the call (thecall is still live), such as in a similar manner as in a ‘digital videorecorder (DVR) for the call. The supervisor may be able to add notes,and even chat live with the agent. Various agent statistics may beshown, such as alerts per call; alerts per day, legal threats, wordsthat trigger, compliance triggers, volume of voice triggers, timingtriggers (too fast/too slow), tempo over time (too slow paced), tempo incurrent call (too fast to be understood), overtalk, and the like.

The system may provide for a user interface, which is also referred toherein as an agent assistant. The user interface may provide for similarinformation as provided in the supervisor's user interface as limited tothe calls for the particular agent. The user interface may also providefor a real-time graphical interface for indicating behavioralcharacteristics of the caller, as well as information for the agent,based on language, categories, scoring and/or acoustic characteristicsof the caller.

The agent assistant may be provided on each agent's desktop to providereal-time alerts, next-best-action guidance to the agent (e.g. links torelevant information or advice when certain items of interest occur onthe call, for example: a technical issue with a product or a specificobjection to an agent offer), and the like. One example of the graphicalindicator of the agent assistant is illustrated in FIG. 9, wherecolor-coded rings surround an alert message area, where the alertmessage area displays messages to the agent, and the colored ringsprovide indicators of how the call is going. For instance, one coloredring may represent a ‘call temperature gauge’, showing through a colorindicator of the emotional state of the caller as based on real-timelanguage and/or acoustics analytics and scoring, such as greenindicating a default condition where the caller is calm and the call isgoing well, orange indicating something has changed either in theacoustic characteristic of the caller's voice or in the language theyare using, and red indicating the caller is now taking actions or has atone that may indicate they are agitated, upset, and the like, and wherethe agent needs to take counter-acting measures. Another colored ringmay represent an alert indicator that indicates the need to take actionsby the agent, such as similarly indicated by different colors, augmentedwith alert messages, and the like. A ring may change color as the toneof the caller's voice changes, such as from green when there wassilence, to yellow as the caller spoke louder, and finally red when thecaller became agitated. For example, after the caller had spoken loudlyand quickly for several seconds, a high agitation warning may be shown,such as by color ring indication and/or a textual alert message. Notonly can the agent's assistant detect agitation/stress and otheremotional states, but it can detect other acoustic events like silenceor over-talk. Through the agent assistant, the agent may receive varioustextual and/or graphical feedback indicators to help them adjust to thechanging behavior of the caller, thus improving the performance of theagent in handling the needs of the caller. In one embodiment, the outerring changes color based on the tone of the call. If the color gets to“red”, it shakes itself to calm down the agent. Acoustically, volume,tone, and agitation (which itself is a measure of tempo, volume, andstress) may be visually displayed in one of the rings, such as the outerring, of the assistant and optionally displayed as the actual score inthe ring such as the center of the ring. Another ring, such as the innerring, may change color based on violations and to provide an alert. Aportion of the mood ring may present an alert message.

In addition to the agent assistant providing behavioral feedback to theuser, such as through color-ring indicators and textual messages, theagent assistant may provide for information in response to key words orphrases from the caller or the absence of key words/phrases, such as ininformation provided in ‘pop-ups’ or ‘fly-outs’. For instance, inresponse to certain things said on the call an informational fly-out mayappear with information relevant to the words that were spoken. Fly-outsmay contain product information, feature information, up-sellsuggestions, companion product suggestions, procedural scripts, links toknowledge databases, data related to the user's account (such as packagetracking, order history, balance due, and the like), and the like. Theymay respond to not only the language on the call, but also to theabsence of language, such as required disclosures that are missing.Fly-outs may contain information related to the agent such as remindersthat required disclosures have not been made, cross-sell/up-sell trendanalysis, tone, performance statistics, and the like. For example, ifthe caller mentions the word “bankruptcy” to a debt collector, aspecific script may be provided to deal with bankrupt debtors. After ascript is provided in a fly-out, the system may continue to analyze thecall, now providing alerts to the agent based on whether or not one ormore portions of the script were spoken.

For example, the agent assistant may be useful in compliance monitoring.With compliance monitoring, agents may be kept from creating veryexpensive finable compliance problems. In a debt collection scenario,the agent must deliver a Mini-Miranda warning within a certain period oftime. If the agent is getting close to missing the window of time todeliver the warning, they are warned of a pending infraction via theagent assistant, such as by a change in color, size, a blink, a pulse, amovement, a signal, a text pop-up, and the like. If the agent violates,they are provided a remedy to deal with the violation. The remedy may bepart of a fly-out or other communication to the agent. Using speechrecognition and category labeling, every call may be tracked in real- ornear real-time for Mini Miranda language, Right Party Contact language,FDCPA violations, abusive language from either party, and other riskylanguage.

In another embodiment, when a call is received at a call center and theparty is identified, a search for data is down with the identifiers. Thesystem is listening and also knows who you are because of a “data dip”.For example, in a call to an e-tailer, when a user asks, “Where's mystuff?” the status of all pending orders are brought out for viewing bythe e-tailer. In another example, in a call to a debtor, if thecollection agent is told that the user is in bankruptcy, the systempulls out a new script rapidly.

There are many powerful use-cases for the RT conversational analyticsfacility. Referring to FIG. 10, a compliance use-case may have thesystem prompt agents to time-sensitive scripts, event-driven procedures,warn agents and monitors of escalating calls, alert agents and monitorsof compliance breeches, and the like, which may result in reducedcomplaints, reduced fines and litigation, improved call efficiency, andthe like. Referring to FIG. 11, a sales use-case may have the systemnotify agents of up-sell or cross-sell opportunities, present targetedoffers based on call metadata and language, guide agents on disclaimersand close procedures, notify supervisors for shadowing opportunities,and the like, which may result in increased revenue, improved callefficiency, reduced disputes, reduced returns, and the like. Referringto FIG. 12, the system may warn agents of increasing escalation ofagitated customers and even offer appropriate talk-down suggestions. Thesystem may alert supervisors immediately to escalation, and offer theopportunity to defuse situations before they get out of hand. This mayresult in reduced transfers, reduced complaints, regulatory appeasement,and the like. Referring to FIG. 13, the system may provide for greatercustomer retention, where the system detects churn language, advisesagents to save procedures and offers or advise a redirect to appropriatere-route to a ‘saves department’, and the like.

FIG. 14 presents an embodiment of a functional diagram for the intakeportion of the RT conversational analytics facility, where calls comingin through the telephony cloud, and end-users (e.g. call agents,supervisors) coming in through LAN/VOIP, provide call portion streams toa call tapper and distributor. Call snippets (e.g., for snippet parsingand analysis, to be fed to a conversational rules facility, and on tothe trigger rules and events facility) and audio slices (e.g., foranalysis for acoustic triggers and trigger rules and events facility)are then input to a real-time mining and rules engine, which feeds theanalytics portion of the system and feeds back events to end usersthrough a web server providing an event feed (e.g., acoustic alerts,conversational alerts, call flow alerts), Web/IIS (e.g. supervisorreal-time data, agent real-time data, call flow data), real-time API(e.g., OEM partners, system alerts), and the like. A database forarchiving events and configurations may be provided, which may then feedback into the real-time mining and rules engine. The system may providefor a contextual real-time facility, such as including an audio slicingalgorithm producing ‘sentence level’ events, acoustic and conversationalpatterns combining for intra-call events and alerts, context confidencethresholds used to suppress costly false positives, a real-time webinterface that pushes alerts to users, and the like. The system mayprovide for workflow facilities, such as including an observer view formonitoring and addressing the most critical events in real-time, anagent view for monitoring recent call performance, a call flow view forreceiving real-time intra-call feedback, and the like. The system mayprovide for volume scaling, such as including capabilities for handlinga large volume of agents (e.g., greater than 1000 agents), a largevolume of channels (e.g., greater than 100 channels), real-timearchitecture capable of supporting re-mining of data if necessary, andthe like. The system may enable expandability, such as including anevent feed as a subscription model that can be used by a third-partyconsumer of real-time business intelligence), a database archive ofevents for historical trending analysis, and the like.

Mining may be facilitated by a search engine. The search engine mayembody a search method and language for identifying a pattern that mayinclude timing (e.g. time lag, time dependencies, inter-word timing)rather than just words and phrases (or Boolean combinations). The searchengine may be a data-type agnostic search engine that can search forspeech parameters, such as time-based speech parameters, on any datasource, regardless of type. The RT conversational analytics facilitycaptures the time-based features of speech. Audio has lags, which placea constraint on how long the text takes to speak, that are lost uponconversion to text typically. Having captured this, a search can be donefor a particular set of words with a minimum lag between them, forexample. In a word flipping search, the search is done for any set ofwords spoken in any order within a time frame. The search enginecombined with the categories (e.g. search-based categories) becomes apatterning engine. The search engine can accommodate searchesconstraints to guarantee words are there or are not there. The searchengine may be scalable and may be cloud-based. In embodiments, speechparameters may be searchable in an Internet search engine. In anembodiment, stand-alone code with no database dependencies may be usedto search data by speech parameters.

In an embodiment, a method for following a customer's journey withrespect to a company/entity regardless of what communication channelthey use (e.g. Calls, Chat, Email, SMS, Twitter, Facebook, Blogs andSurveys) may be enabled. From any customer contact, users may be able tonavigate to previous or subsequent contacts from the same customer fromany channel/platform. When contact linking is combined with RepeatContact rules, it can be a very powerful tool for determining root causeof repeat contacts and poor first contact resolution, even when contactscross channels. For example, a user may use Twitter to get an airline'ssupport. The airline can track repeat communication via the Twitter ID.Then, the user may call and talk to the airline about something else.Then, the user may post a blog about the airline. These threecommunications can be tracked together. Scores may be mapped so thatthey become “super-interaction” scores or a score of loyalty. Sincecustomers can touch the entity in a multi-channel way, the RTconversational analytics facility is uniquely adapted to unify thesechannels.

In recent years there has been increasing regulation relative topersonally identifiable information (PII) such as date and place ofbirth, mother's maiden name, biometric records, medical, educational,financial, and employment information, social security number, creditcard data, financial account information and the like. State and federallegislation has been enacted requiring business which receive and storesuch information to provide increased security with respect to suchinformation such as logging all individuals having access to personallyidentifiable information, limiting the distribution of such information,encrypting the information, limiting accessibility to such information,and the like. Business may find these regulations impose an increasingcost burden. In some cases it may be more desirable to simply remove ordelete any personally identifiable information from stored data.Businesses are not allowed to store PCI sensitive information such asaccount numbers or security codes. The redaction capability, by usingconversational and acoustic analysis, can heuristically identifymillions of possible combinations of potentially sensitive data and tagthem. Additionally, instead of having a few dozen patterns, there areseveral hundred, each of them created and tested against a very largeset of real data. Finally, when a “potential” area is identified, otherpatterns are deployed to ensure accuracy and overlap. For instance, if accv code mention is found, not only are the numbers removed, butexpiration date and credit card numbers are explored for and identifiedfor removal. In a sixteen-digit credit card, removing any four numbersmakes it virtually impossible to exploit the credit card and removingany other data, like ccv codes or expiration dates eliminates possiblefraud. In a large-scale test with an auditing body, no exploitableinformation was retained after the redaction process.

PCI Targeted Redaction redacts numbers associated with specific types ofPCI data. Redaction is performed based on locating keywords in thetranscripts, and redacting numbers near those keywords.

Referring to FIG. 19, a top-level process flow diagram 1900 is presentedcomprising receiving an incoming vocal stream 1902, converting incomingvocal stream to text 1904, analyzing the text for sensitive information1908 according to a set of rules for analysis 1912, identifying thelocation of the sensitive information 1910, redacting sensitiveinformation from the vocal stream 1914 and outputting a redacted vocalstream 1918. In embodiments, the incoming vocal stream may comprise atelephonic voice communication. In embodiments, converting vocal streamto text 1904 may occur in real time as the vocal stream is beingacquired, with a minimal delay, from existing or archived vocal data,and the like. Conversion of the incoming vocal stream to text 1904 maybe accomplished using one or more of large vocabulary speech recognitiontechnology, phonetic engines, direct phrase recognition, cloud-basedspeech recognition, and the like. After converting the vocal stream totext 1904, the system may analyze the text for sensitive information1908 according to a set of rules for analysis 1912. In embodiments, thesystem may receive incoming text 1920 that the system may analyze forsensitive information 1908 according to a set of rules for analysis1912. As the analysis of the text identifies sensitive information,identification of the location of the sensitive information in the vocalstream 1910 occurs. The location of sensitive data in the recognizedcommunications is identified and “tagged” using an exhaustive list oflanguage “patterns” (credit card strings, expiration dates, ccv codes,social security numbers, and a multitude of other PII data). The startand stop locations/times are annotated and passed on for redaction.Sensitive information at the identified location is redacted 1914 andthe redacted vocal stream is output 1918. Sensitive information may beredacted from the text 1922 and redacted text output 1924. Inembodiments, the redacted text output 1924 and the redacted vocal streamoutput 1918 may be stored in a common searchable database. Using thetagged locations, the sensitive audio is replaced withsilence/muted/bleeped. This newly redacted file is then encrypted andwritten to a cache on a storage device or written back into the sourceformat and repository. In embodiments, the tagged locations may beapplied to the stream of text before it is to be parsed and written intothe database. This text is either the output of the speech to textengine or the direct text coming from a text-based channel. The taggedwords may be removed entirely or substituted for other placeholder wordsthat can be recognized.

In embodiments, rules of analysis 1912 may comprise key words andphrases, the presence of numbers or series of numbers, and the like. Inembodiments, the rules of analysis 1912 may comprise rules such as:automatically redact all numbers where two or more numbers are found insequence; automatically redact numbers identified as being separated byless than a specified number of words (e.g. any two numbers within oneword of each other are redacted, and redact intervening single wordsbetween the two numbers); automatically redact numbers within aspecified number of words following a key word or phrase, and the like.Various examples will be described. In a first example, the transcriptreads “Cost is sixty nine dollars and ninety nine cents”. Afterredaction, the phrase becomes “Cost is . . . dollars and . . . cents”.All numbers redacted due to being within one word of each other—the restof the words remain. In a second example, the transcript reads “Cardnumber is five three uh two one four”. After redaction, the phrasebecomes “Card number is . . . ”. All numbers and the word “uh” areredacted due to being within one word of each other. In a third example,the transcript reads “Card number is five three uh uh two one four”.After redaction, the phrase becomes “Card number is . . . uh uh . . . ”.The words “uh uh” remain due to being two words between numbers. In afourth example, the transcript reads “Card number is five ate blankthree two”. After redaction, it becomes “Card number is five ate blank .. . ”. The word “five” is not within one word of another number so it isnot redacted. In a fifth example, the transcript reads “We will give youa seven day trial”. After redaction, it becomes “We will give you aseven day trial”. Here, there is only one number so no redaction occurs.In a sixth example, the transcript reads “We will give you a seventyfive day trial”. After redaction, it reads “We will give you a . . . daytrial”. The numbers are redacted due to there being two of them.

In embodiments, redaction of sensitive information at identifiedlocations 1914 may comprise one or more of: removing that portion of thevocal stream; replacing identified sensitive information in the vocalstream with silence (muting); adding additional audible noise, bleeping,to the vocal stream at the location of the identified sensitiveinformation; and the like. Redaction of said sensitive information mayfacilitate compliance with regulations such as those of the AffordableCare Act (ACA), the Health Insurance Portability and Accountability Act(HIPPA), Consumer Financial Protection Bureau (CFPB), Fair DebtCollection Practices Act (FDCPA), data privacy regulations, and thelike.

Referring to FIG. 20, a high level embodiment of rules of analysis 1912is shown comprising: a table comprising a plurality of key word/phrase2002 entries and corresponding number of words searched 2004 entries; alist of introductory phrases 2008; and the like. In embodiments, thesystem may identify a key word/phrase 2002 from the table and redactinformation, such as numbers, found within the specified number of wordssearched 2004 following the key word/phrase 2002 specified in the table.The number of words searched 2004 may vary with the key word/phrase2002. In embodiments, it may be possible for the rules of analysis 1912to be customized to identify customer specific sensitive information tobe redacted.

In embodiments, the rules of analysis 1912 may be customized to comprisecontextual clues to privileged conversations that may be redacted.Privileged conversations may include those between a lawyer and theirclient, between physicians or psychologists and their patients includingcontent such as names, diagnosis, and the like. In embodiments, theremay be a list of introductory phrases 2008 to facilitate contextualanalysis wherein the introductory phrase may be followed by sensitive,privileged information to be redacted.

In an illustrative and non-limiting example, FIG. 20 shows a table ofkey word/phrase 2002 entries and corresponding number of words search2004 entries. During analysis, the system may identify “Credit CardNumber” 2002A. The system may then redact any numbers within the ‘40’2004A following words of the identified phrase “Credit Card Number”2002A. During analysis, the system may identify “Diagnosis” 2008A as anintroductory phrase 2008. Information following the introductory phrase“Diagnosis” 2008A may be further contextually analyzed and redacted asappropriate.

In embodiments, a specific recording device (aka: a tapper) may beadapted for realtime analytics that first analyzes the audio and thenrecords/persists to long-term memory or does not record to long termbased on that analysis/contextual processing. Redaction is a user casefor the tapper.

In embodiments, the identification and redaction of sensitiveinformation may be designed to facilitate compliance with the PaymentCard Industry (PCI) standards while retaining other numbers in the textsuch as pricing and volume data. In one embodiment, a PCI Redactorproduct may be speech analytics technology to automaticallyremove/mute/bleep PCI sensitive authentication and cardholder data inaccordance with one or more rules from incoming call recordings or callrecording archives. Information such as credit card number, securitycode, expiration date, PIN number, date of birth, driver's license data,social security number, account data, and the like may be redacted tofacilitate compliance. In embodiments, keywords may be located in thetranscript of an audio call and numbers redacted based on a set of rulesof analysis 1912. Rules of analysis 1912 may comprise one or more of thefollowing: redacting numbers in close proximity to identified keywords/phrases 2002 (e.g. any two numbers within two words of each otherare redacted); iterative processing of text following redacted numbersand further redaction of numbers (e.g. After the initial redaction hasbeen performed, a second pass is done, looking at the next 10 words fromthe last redacted number, and if numbers are found, these are alsoredacted. Then, a third pass is then done to identify if there is onemore number in the sequence.); not redacting numbers which are closelyfollowed by units not consistent with PCI information such as dollar(s),cent(s), buck(s), minutes, hour(s), day(s), week(s), month(s), monthly,year(s), channel(s), percent(s), percentage, refundable, nonrefundable,promotion, promotional, processing, fee(s), total(s), and the like toassist in keeping things like prices and durations from being redacted;and the like.

In a first example, the transcript may read “what is the card number youwill be using today oh let me find my card it is in my wallet take yourtime my purse is always such a mess let me see here oh here it is thenumber is five four three one seven ate zero two heaven uh seven fournine two six thank you now what is the expiration date . . . ” Thephrase “Card number” is the keyword identified in the transcript as aclue that information that may need to be redacted may follow. Thus, apre-determined number of words following the phrase “card number” aresearched for numbers to be redacted. In an embodiment, the number ofwords searched may range from 15 to 40 words or so. In this example, thenumbers “five four three one” are redacted, due to ‘one’ being within 2words of ‘five’. Then, the next 10 words after the last redacted word(‘one’) are searched to find numbers. The words/numbers “seven ate zerotwo” are redacted, as are “seven four nine two”. The next word is lookedat and since it is a number (six′), it is then redacted. The finaltranscript after redaction reads “what is the card number you will beusing today oh let me find my card it is in my wallet take your time mypurse is always such a mess let me see here oh here it is the number isredacted redacted heaven uh redacted redacted thank you now what is theexpiration date . . . .”

In a second example, the transcript may read “what is the expirationdate oh nine sixteen thank you we will be charging this card ninety ninedollars and twenty ur uh three cents on Friday . . . ” The word“expiration” is the keyword identified in the transcript as a clue thatinformation that may need to be redacted may follow. Thus, apre-determined number of words following the phrase “expiration” aresearched for numbers to be redacted. In an embodiment, the number ofwords searched may range from 15 to 40 words or so. One or more chartsof number of words searched for each keyword may be provided andconsulted by the redaction process—the one or more charts may becustomizable by a user of the system. The numbers “oh nine sixteen” areredacted. The numbers “ninety nine” are not redacted because of beingfollowed by ‘dollars’. The numbers “twenty three” are not redactedbecause they are followed (within 3 words) by ‘cents’. The finaltranscript after redaction reads “what is the expiration date redactedthank you we will be charging this card ninety nine dollars and twentyur uh three cents on Friday . . . ” In embodiments, a string of numbersmay be removed based on endpointing (based on silences).

In embodiments, audio data, such as that of a phone call, may beintercepted at the source and redacted prior to being promulgatedthrough the system and made accessible/audible by other participants ina phone call. In embodiments, this may be used for fraud protection as asafeguard to limit the ability of certain parties to provide sensitiveinformation over the phone, and the like. In some embodiments, thetransmission of sensitive PCI information may be permitted to a selectlist of recipients, and redacted for all other participants.

Referring to FIG. 21, a high level flow diagram 2100 illustrates anembodiment of source redaction system comprising a user 2102 at thesource, identifying one or more recipient(s) of a vocal stream 2104, alist of approved recipients 2110, alternate paths to a recipient 2118which may be selected based on whether the recipient is on the list ofapproved recipient(s) 2108. In embodiments, if the recipient 2118 is onthe list of approved recipients 2110, an outgoing unredacted vocalstream 2112 may be sent through the telephony infrastructure 2114 to therecipient 2118. If the recipient 2118 is not on the list of approvedrecipients 2110, sensitive information may be redacted from the vocalstream and an outgoing redacted vocal stream 1918 may be sent to therecipient 2118. The telephony infrastructure 2114 may comprise one ormore of Voice over IP (VOIP) traditional PBX, and the like.

In an illustrative and non-limiting embodiment, a user 2102 is on thetelephone and is asked for confidential information such as credit carddata, bank account data, and the like. If a recipient, 2118 is on a listof approved recipients 2110, the sensitive information may betransmitted without redaction and the recipient 2118 will be able tohear the sensitive information. If a recipient, 2110, is not on the listof approved recipients, 2110, sensitive information may be redacted fromthe vocal stream through removal, muting, beeping or the like. Theresultant outgoing redacted vocal stream 1918 may be sent to therecipient 2110B who may not be able to hear the sensitive information asit has been redacted prior to being transmitted to them.

In embodiments, this source redaction capability may be incorporatedinto a phone, a local home network system, and the like. It may be usedto limit the exposure of elderly persons, children, and the like topredatory phone solicitations, allowing them to share sensitiveinformation only with approved recipients. In embodiments, the approvedrecipients may be included in a whitelist of a profile controlling theredaction capability. Thus, only whitelisted entities can have a creditcard number or account number spoken to them by an elder.

In embodiments, there may be a recording device capable of real timeredaction which caches the incoming vocal stream in temporary memory,analyzes the incoming vocal stream, redacts the vocal data streamaccording to set of rules of analysis 1912 and then records/persists theoutgoing redacted vocal stream 1918 to long-term memory.

In embodiments, sensitive information such as PCI information, cardholder data, and the like may be removed from the audio stream in thecache prior to being recorded for call archiving, and the like. Inembodiments, the audio stream may be split making available anun-redacted version to an agent and a redacted version for the callarchiving system. Thus, in an illustrative and non-limiting example, anagent may be able to hear payment information and enter it on a form andyet the sensitive information is not recorded as part of the callarchive. This may reduce the security level required for the callarchives, as they may not contain sensitive PCI information.

Referring to FIG. 22A, a high level flow diagram 2200 is shownillustrating a process flow comprising: caching an incoming vocal stream2202; redacting sensitive information from the vocal stream; sending anunredacted vocal stream to an agent 2208; and recording a redacted vocalstream 2204. In embodiments, an incoming vocal stream 1902 may bereceived through standard telephony infrastructure 2114. The system maycache the vocal stream 2202 and may then send an unredacted vocal streamto an agent 2208 who may process the call. While the vocal stream iscached, it may be converted to text, the text analyzed for sensitiveinformation and the sensitive information redacted from the cached vocalstream. The system may record the redacted vocal stream 2204. Thus, asdescribed in this embodiment, sensitive information may be madeavailable to an agent for processing but redacted prior to beingrecorded.

In embodiments, sensitive information such as PCI information, cardholder data, and the like may be removed from the audio stream in thecache prior to being recorded for call archiving, or being sent to anagent. In embodiments, when upcoming PCI data is identified, such ascontextually identified, the caller or user may be routed to analternate method of providing payment information. Referring to FIG.22B, a high level flow diagram 2290 is shown illustrating a process flowcomprising: caching an incoming vocal stream 2202; redacting sensitiveinformation from the vocal stream; recording the redacted vocal stream2204; sending a redacted vocal stream to an agent 2210; and redirectingthe unredacted vocal stream to alternate information gathering methods2212.

In embodiments, a data server may deploy the speech analyticstechnologies described above to identify PCI sensitive authenticationand cardholder data. Incoming vocal streams may be stored in a temporarybuffer on the data server and sensitive information redacted prior totransferring the redacted vocal stream to permanent storage, a callagent, and the like.

In embodiments, redaction may be multi-level where the redaction may bedone multiple times and in a plurality of locations such as at thesource, in the cloud, during transmission, at the receiving location, onarchived data and the like. In an illustrative and non-limiting example,a vocal stream may initially be redacted at its source, such as at thephone of a caller. A secondary analysis and redaction may occur as theincoming vocal stream is processed in a call center. In embodiments,additional analysis and redaction may occur on the data stored in a callrecording archive.

Thus, the redaction capability operates on advanced speech analyticstechnology, delivering accurate and reliable results. The redactioncapability enables the maintenance of PCI compliant call recording andcertification and the reduction of privacy risk by removing sensitivedata from call recordings. The redaction capability enables redaction oftargeted content recorded from call connection through the IVR, hold,transfer, conversation and post-close survey. The redaction capabilitycan be used to certify redaction from call recording archives. Theredaction capability does not depend on a change in payment processing,human intervention, or integration with CRM systems.

All of the systems and methods described herein may be useful infacilitating realtime calls. In collections, the systems and methods maysupport inbound and outbound sales support. In knowledge/technicalsupport, a help desk agent may be able to pull up data/productsassociated with an articulated issue automatically based on recognitionsand categories. Industries with compliance-related issues, customerservice, technical support, and the like may find the systems andmethods useful. In agent performance monitoring, the systems and methodsmay be used for cross-sell/up-sell trend analysis, competitiveintelligence, call center automated data integration, compliance, andthe like. With compliance, the CPFB requires tone monitoring and theCFTC requires that audio be discoverable, both of which are possiblewith the systems and methods described herein.

In embodiments, these devices and methods may be integrated into largercustomer systems such as those used by banks, credit unions, credit cardcompanies, traders, brokerage firms, large retailers, collectionagencies, and the like. In embodiments, these devices and methods may behoused and implemented entirely at a client site.

Various auxiliary products, such as screen capture, audio capture, andsimultaneous screen capture with call recording may be possible with thesystems and methods.

In embodiments, the systems and methods described herein may be deployedas a mobile application or as a device-specific application.

In an embodiment, an application may allow the caller/consumer to obtaina transcript of the call once they have hung up.

In embodiments, any of the methods, steps or processes described hereinmay occur automatically, under computer control, or in a time spanshorter than a browser, webpage, or application refresh or webpage load.This time span may be less than about 10 sec, less than about 5 sec,less than about 1 second, or substantially instantaneously.

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software, program codes,and/or instructions on a processor. The present invention may beimplemented as a method on the machine, as a system or apparatus as partof or in relation to the machine, or as a computer program productembodied in a computer readable medium executing on one or more of themachines. The processor may be part of a server, client, networkinfrastructure, mobile computing platform, stationary computingplatform, or other computing platform. A processor may be any kind ofcomputational or processing device capable of executing programinstructions, codes, binary instructions and the like. The processor maybe or include a signal processor, digital processor, embedded processor,microprocessor or any variant such as a co-processor (math co-processor,graphic co-processor, communication co-processor and the like) and thelike that may directly or indirectly facilitate execution of programcode or program instructions stored thereon. In addition, the processormay enable execution of multiple programs, threads, and codes. Thethreads may be executed simultaneously to enhance the performance of theprocessor and to facilitate simultaneous operations of the application.By way of implementation, methods, program codes, program instructionsand the like described herein may be implemented in one or more thread.The thread may spawn other threads that may have assigned prioritiesassociated with them; the processor may execute these threads based onpriority or any other order based on instructions provided in theprogram code. The processor may include memory that stores methods,codes, instructions and programs as described herein and elsewhere. Theprocessor may access a storage medium through an interface that maystore methods, codes, and instructions as described herein andelsewhere. The storage medium associated with the processor for storingmethods, programs, codes, program instructions or other type ofinstructions capable of being executed by the computing or processingdevice may include but may not be limited to one or more of a CD-ROM,DVD, memory, hard disk, flash drive, RAM, ROM, cache and the like.

A processor may include one or more cores that may enhance speed andperformance of a multiprocessor. In embodiments, the process may be adual core processor, quad core processors, other chip-levelmultiprocessor and the like that combine two or more independent cores(called a die).

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software on a server,client, firewall, gateway, hub, router, or other such computer and/ornetworking hardware. The software program may be associated with aserver that may include a file server, print server, domain server,internet server, intranet server and other variants such as secondaryserver, host server, distributed server and the like. The server mayinclude one or more of memories, processors, computer readable media,storage media, ports (physical and virtual), communication devices, andinterfaces capable of accessing other servers, clients, machines, anddevices through a wired or a wireless medium, and the like. The methods,programs, or codes as described herein and elsewhere may be executed bythe server. In addition, other devices required for execution of methodsas described in this application may be considered as a part of theinfrastructure associated with the server.

The server may provide an interface to other devices including, withoutlimitation, clients, other servers, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more location without deviating from the scope ofthe invention. In addition, any of the devices attached to the serverthrough an interface may include at least one storage medium capable ofstoring methods, programs, code and/or instructions. A centralrepository may provide program instructions to be executed on differentdevices. In this implementation, the remote repository may act as astorage medium for program code, instructions, and programs.

The software program may be associated with a client that may include afile client, print client, domain client, internet client, intranetclient and other variants such as secondary client, host client,distributed client and the like. The client may include one or more ofmemories, processors, computer readable media, storage media, ports(physical and virtual), communication devices, and interfaces capable ofaccessing other clients, servers, machines, and devices through a wiredor a wireless medium, and the like. The methods, programs, or codes asdescribed herein and elsewhere may be executed by the client. Inaddition, other devices required for execution of methods as describedin this application may be considered as a part of the infrastructureassociated with the client.

The client may provide an interface to other devices including, withoutlimitation, servers, other clients, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more location without deviating from the scope ofthe invention. In addition, any of the devices attached to the clientthrough an interface may include at least one storage medium capable ofstoring methods, programs, applications, code and/or instructions. Acentral repository may provide program instructions to be executed ondifferent devices. In this implementation, the remote repository may actas a storage medium for program code, instructions, and programs.

The methods and systems described herein may be deployed in part or inwhole through network infrastructures. The network infrastructure mayinclude elements such as computing devices, servers, routers, hubs,firewalls, clients, personal computers, communication devices, routingdevices and other active and passive devices, modules and/or componentsas known in the art. The computing and/or non-computing device(s)associated with the network infrastructure may include, apart from othercomponents, a storage medium such as flash memory, buffer, stack, RAM,ROM and the like. The processes, methods, program codes, instructionsdescribed herein and elsewhere may be executed by one or more of thenetwork infrastructural elements.

The methods, program codes, and instructions described herein andelsewhere may be implemented on a cellular network having multiplecells. The cellular network may either be or include a frequencydivision multiple access (FDMA) network or a code division multipleaccess (CDMA) network. The cellular network may include mobile devices,cell sites, base stations, repeaters, antennas, towers, and the like.The cell network may be one or more of GSM, GPRS, 3G, EVDO, mesh, orother network types.

The methods, programs codes, and instructions described herein andelsewhere may be implemented on or through mobile devices. The mobiledevices may include navigation devices, cell phones, mobile phones,mobile personal digital assistants, laptops, palmtops, netbooks, pagers,electronic books readers, music players and the like. These devices mayinclude, apart from other components, a storage medium such as a flashmemory, buffer, RAM, ROM and one or more computing devices. Thecomputing devices associated with mobile devices may be enabled toexecute program codes, methods, and instructions stored thereon.Alternatively, the mobile devices may be configured to executeinstructions in collaboration with other devices. The mobile devices maycommunicate with base stations interfaced with servers and configured toexecute program codes. The mobile devices may communicate on apeer-to-peer network, mesh network, or other communications network. Theprogram code may be stored on the storage medium associated with theserver and executed by a computing device embedded within the server.The base station may include a computing device and a storage medium.The storage device may store program codes and instructions executed bythe computing devices associated with the base station.

The computer software, program codes, and/or instructions may be storedand/or accessed on machine readable media that may include: computercomponents, devices, and recording media that retain digital data usedfor computing for some interval of time; semiconductor storage known asrandom access memory (RAM); mass storage typically for more permanentstorage, such as optical discs, forms of magnetic storage like harddisks, tapes, drums, cards and other types; processor registers, cachememory, volatile memory, non-volatile memory; optical storage such asCD, DVD; removable media such as flash memory (e.g. USB sticks or keys),floppy disks, magnetic tape, paper tape, punch cards, standalone RAMdisks, Zip drives, removable mass storage, off-line, and the like; othercomputer memory such as dynamic memory, static memory, read/writestorage, mutable storage, read only, random access, sequential access,location addressable, file addressable, content addressable, networkattached storage, storage area network, bar codes, magnetic ink, and thelike.

The methods and systems described herein may transform physical and/oror intangible items from one state to another. The methods and systemsdescribed herein may also transform data representing physical and/orintangible items from one state to another.

The elements described and depicted herein, including in flow charts andblock diagrams throughout the figures, imply logical boundaries betweenthe elements. However, according to software or hardware engineeringpractices, the depicted elements and the functions thereof may beimplemented on machines through computer executable media having aprocessor capable of executing program instructions stored thereon as amonolithic software structure, as standalone software modules, or asmodules that employ external routines, code, services, and so forth, orany combination of these, and all such implementations may be within thescope of the present disclosure. Examples of such machines may include,but may not be limited to, personal digital assistants, laptops,personal computers, mobile phones, other handheld computing devices,medical equipment, wired or wireless communication devices, transducers,chips, calculators, satellites, tablet PCs, electronic books, gadgets,electronic devices, devices having artificial intelligence, computingdevices, networking equipment, servers, routers and the like.Furthermore, the elements depicted in the flow chart and block diagramsor any other logical component may be implemented on a machine capableof executing program instructions. Thus, while the foregoing drawingsand descriptions set forth functional aspects of the disclosed systems,no particular arrangement of software for implementing these functionalaspects should be inferred from these descriptions unless explicitlystated or otherwise clear from the context. Similarly, it will beappreciated that the various steps identified and described above may bevaried, and that the order of steps may be adapted to particularapplications of the techniques disclosed herein. All such variations andmodifications are intended to fall within the scope of this disclosure.As such, the depiction and/or description of an order for various stepsshould not be understood to require a particular order of execution forthose steps, unless required by a particular application, or explicitlystated or otherwise clear from the context.

The methods and/or processes described above, and steps thereof, may berealized in hardware, software or any combination of hardware andsoftware suitable for a particular application. The hardware may includea general-purpose computer and/or dedicated computing device or specificcomputing device or particular aspect or component of a specificcomputing device. The processes may be realized in one or moremicroprocessors, microcontrollers, embedded microcontrollers,programmable digital signal processors or other programmable device,along with internal and/or external memory. The processes may also, orinstead, be embodied in an application specific integrated circuit, aprogrammable gate array, programmable array logic, or any other deviceor combination of devices that may be configured to process electronicsignals. It will further be appreciated that one or more of theprocesses may be realized as a computer executable code capable of beingexecuted on a machine-readable medium.

The computer executable code may be created using a structuredprogramming language such as C, an object oriented programming languagesuch as C++, or any other high-level or low-level programming language(including assembly languages, hardware description languages, anddatabase programming languages and technologies) that may be stored,compiled or interpreted to run on one of the above devices, as well asheterogeneous combinations of processors, processor architectures, orcombinations of different hardware and software, or any other machinecapable of executing program instructions.

Thus, in one aspect, each method described above and combinationsthereof may be embodied in computer executable code that, when executingon one or more computing devices, performs the steps thereof. In anotheraspect, the methods may be embodied in systems that perform the stepsthereof, and may be distributed across devices in a number of ways, orall of the functionality may be integrated into a dedicated, standalonedevice or other hardware. In another aspect, the means for performingthe steps associated with the processes described above may include anyof the hardware and/or software described above. All such permutationsand combinations are intended to fall within the scope of the presentdisclosure.

While the invention has been disclosed in connection with the preferredembodiments shown and described in detail, various modifications andimprovements thereon will become readily apparent to those skilled inthe art. Accordingly, the spirit and scope of the present invention isnot to be limited by the foregoing examples, but is to be understood inthe broadest sense allowable by law.

All documents referenced herein are hereby incorporated by reference.

What is claimed is:
 1. A system for evaluating one or more portions of acommunication, comprising: a language module structured to analyze theone or more portions of the communication for at least one of a languagecharacteristic or a non-word symbol within a text of the communication,wherein the language characteristic is at least one of a category, asentiment, a regulation compliance, a score, a text, an alternativetext, a presence or absence of specific language, a phoneme, a word, aphrase, a placeholder word used in redaction, a language pattern or aconfidence in word match; a metadata module that acts as a filter to theone or more portions of the communication; an acoustic analysis modulestructured to analyze the one or more portions of the communication foran acoustic characteristic, wherein the acoustic characteristic is atleast one of a stress of words, an aggregated stress of a plurality ofwords, an emphasis, an agitation, a tempo, a change in tempo, an amountof silence, a silence between words, a gain in volume or energy of thewords, a tone, an overtalk, a time lag between words or symbols, a timedependency between key words or phrases, an inter-word timing, aninter-word sequencing, an inter-word distance, an inflexion of words, ora temporal pattern; and a determination module structured to determineat least one category label to associate with the one or more portionsof the communication using at least one of the language characteristicor the non-word symbol, and the acoustic characteristic.
 2. The systemof claim 1, wherein the language pattern is an order of occurrence ofportions of the text.
 3. The system of claim 1, wherein the languagepattern is an order of occurrence of portions of the text within a timeframe.
 4. The system of claim 1, wherein the language pattern is aBoolean relationship of portions of the text.
 5. The system of claim 4,wherein the Boolean relationship is AND, OR, or NOT.
 6. The system ofclaim 1, wherein the language pattern is a specific location in the oneor more portions of the communication.
 7. The system of claim 1, whereinthe language pattern is a proximity of a topic.
 8. The system of claim1, wherein the at least one category label comprises a combination ofone or more other category labels.
 9. The system of claim 1, wherein ifno category label can be associated with the one or more portions of thecommunication, the determination module is structured to generate a newcategory label and add the new category label to a plurality of categorylabels for a subsequent determination.
 10. The system of claim 1,wherein the at least one category label is at least one of a behavior, areason, a procedure, a competitor, a dissatisfaction, an empathy, arepeat contact, a transferred call or a politeness or a unique labelassigned by the user or the system.
 11. The system of claim 10, whereina numerical weight is user-defined for the at least one category label,a plurality of weights for all category labels for the communication areaggregated, and a determination is made when the aggregation exceeds athreshold.
 12. The system of claim 11, wherein the determination enablestracking progress towards a goal.
 13. The system of claim 1, wherein theone or more portions of the communication are at least one of a voicecommunication, a voice mail, a chat, an email, a blog post, a survey, asocial media post, an IVR session, or an SMS.
 14. The system of claim13, wherein the voice communication is selected from the groupconsisting of VoIP, TDM and SIP.
 15. The system of claim 1, wherein whenthe one or more portions of the communication are a voice communication,the system is adapted to convert the voice communication to text using acomputer-based speech recognition module before analyzing the voicecommunication for the language characteristic.
 16. The system of claim1, wherein the metadata module further determines a metadata of thecommunication, wherein the metadata are at least one of a speaker name,a speaker type, communication type, an agent data, an agent grouping, acall handling location, a time and date of call, a department, a skillor queue, a pertinent IVR path data, another category or not anothercategory, or a call length.
 17. The system of claim 1, wherein thenon-word symbol is at least one of an emoticon, a capitalization, apunctuation, an ellipsis, a non-alphanumeric character, or a spacing.18. The system of claim 1, further comprising, a contextual modulestructured to analyze a context of the one or more portions of thecommunication, wherein the determination module is further structured todetermine the at least one category label associated with the one ormore portions of the communication using the context.
 19. The system ofclaim 18, wherein the contextual module is structured to examine the oneor more portions of the communication for specific languagecharacteristics, non-word symbols, or acoustic characteristics thattrigger potential contextual relevance to subsequent or prior specificlanguage characteristics, non-word symbols, or acoustic characteristicsof the one or more portions of the communication.